[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 737852338 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2400.000 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001013] APIC: Switch to symmetric I/O mode setup [ 0.002332] x2apic enabled [ 0.003000] Switched APIC routing to physical x2apic. [ 0.003016] kvm-guest: setup PV IPIs [ 0.006364] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 0.007024] Calibrating delay loop (skipped) preset value.. 4800.00 BogoMIPS (lpj=2400000) [ 0.008012] pid_max: default: 32768 minimum: 301 [ 0.010283] LSM: Security Framework initializing [ 0.011063] Yama: becoming mindful. [ 0.012047] SELinux: Initializing. [ 0.013076] *** VALIDATE selinux *** [ 0.022846] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026824] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028072] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029118] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030130] *** VALIDATE tmpfs *** [ 0.031504] *** VALIDATE proc *** [ 0.032291] *** VALIDATE cgroup *** [ 0.033022] *** VALIDATE cgroup2 *** [ 0.034293] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035163] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037035] Spectre V2 : User space: Vulnerable [ 0.038014] Speculative Store Bypass: Vulnerable [ 0.041421] debug: unmapping init [mem 0xffffffffa5a59000-0xffffffffa5a60fff] [ 0.044282] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045718] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046027] ... version: 2 [ 0.047014] ... bit width: 48 [ 0.048016] ... generic registers: 4 [ 0.049014] ... value mask: 0000ffffffffffff [ 0.050029] ... max period: 00007fffffffffff [ 0.051048] ... fixed-purpose events: 3 [ 0.052015] ... event mask: 000000070000000f [ 0.053286] rcu: Hierarchical SRCU implementation. [ 0.055478] smp: Bringing up secondary CPUs ... [ 0.056697] x86: Booting SMP configuration: [ 0.057027] .... node #0, CPUs: #1 #2 #3 [ 0.070181] smp: Brought up 1 node, 4 CPUs [ 0.072028] smpboot: Max logical packages: 1 [ 0.073016] smpboot: Total of 4 processors activated (19200.00 BogoMIPS) [ 0.114520] node 0 deferred pages initialised in 39ms [ 0.118596] devtmpfs: initialized [ 0.119253] x86/mm: Memory block size: 128MB [ 0.122080] gcov: version magic: 0x41383552 [ 0.124305] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.125081] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.126397] pinctrl core: initialized pinctrl subsystem [ 0.127212] [ 0.127999] ************************************************************* [ 0.128018] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.129019] ** ** [ 0.130018] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.131017] ** ** [ 0.132022] ** This means that this kernel is built to expose internal ** [ 0.133019] ** IOMMU data structures, which may compromise security on ** [ 0.134017] ** your system. ** [ 0.135019] ** ** [ 0.136017] ** If you see this message and you are not debugging the ** [ 0.137018] ** kernel, report this immediately to your vendor! ** [ 0.138076] ** ** [ 0.139017] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.140016] ************************************************************* [ 0.141884] NET: Registered protocol family 16 [ 0.142571] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.143066] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.144065] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.146126] cpuidle: using governor menu [ 0.148822] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.152701] PCI: Using configuration type 1 for base access [ 0.155199] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.165098] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.166000] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.169144] cryptd: max_cpu_qlen set to 1000 [ 0.172110] ACPI: Added _OSI(Module Device) [ 0.174020] ACPI: Added _OSI(Processor Device) [ 0.176014] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.178014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.182857] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.192101] ACPI: Interpreter enabled [ 0.193076] ACPI: PM: (supports S0 S3 S4 S5) [ 0.194013] ACPI: Using IOAPIC for interrupt routing [ 0.195134] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.196454] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.209036] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.210044] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.211028] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.212093] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.217076] acpiphp: Slot [2] registered [ 0.218164] acpiphp: Slot [5] registered [ 0.219164] acpiphp: Slot [6] registered [ 0.220139] acpiphp: Slot [7] registered [ 0.221161] acpiphp: Slot [8] registered [ 0.222143] acpiphp: Slot [9] registered [ 0.223110] acpiphp: Slot [10] registered [ 0.224146] acpiphp: Slot [3] registered [ 0.225165] acpiphp: Slot [4] registered [ 0.226136] acpiphp: Slot [11] registered [ 0.227128] acpiphp: Slot [12] registered [ 0.228153] acpiphp: Slot [13] registered [ 0.229139] acpiphp: Slot [14] registered [ 0.230312] acpiphp: Slot [15] registered [ 0.231107] acpiphp: Slot [16] registered [ 0.232764] acpiphp: Slot [17] registered [ 0.233122] acpiphp: Slot [18] registered [ 0.234092] acpiphp: Slot [19] registered [ 0.235127] acpiphp: Slot [20] registered [ 0.236397] acpiphp: Slot [21] registered [ 0.237133] acpiphp: Slot [22] registered [ 0.238214] acpiphp: Slot [23] registered [ 0.239357] acpiphp: Slot [24] registered [ 0.240122] acpiphp: Slot [25] registered [ 0.241137] acpiphp: Slot [26] registered [ 0.242117] acpiphp: Slot [27] registered [ 0.243217] acpiphp: Slot [28] registered [ 0.244099] acpiphp: Slot [29] registered [ 0.245104] acpiphp: Slot [30] registered [ 0.246169] acpiphp: Slot [31] registered [ 0.247177] PCI host bridge to bus 0000:00 [ 0.248024] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.249026] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.250024] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.251029] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.252033] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.253175] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.256255] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.262057] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.267313] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.285015] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.292000] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.298021] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.303017] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.307019] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.313185] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.318061] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.322713] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.329723] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.340015] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.366018] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.378020] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.387252] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.399000] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.410017] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.453084] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.468000] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.476141] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.492021] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.513016] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.531182] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.543136] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.554020] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.579020] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.598103] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.605017] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.623017] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.652018] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.665314] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.672022] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.680020] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.718019] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.736248] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.749000] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.763019] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.798021] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.817000] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.818367] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.820506] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.822501] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.825362] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.829234] iommu: Default domain type: Passthrough [ 0.831593] SCSI subsystem initialized [ 0.834176] ACPI: bus type USB registered [ 0.836296] usbcore: registered new interface driver usbfs [ 0.840136] usbcore: registered new interface driver hub [ 0.843153] usbcore: registered new device driver usb [ 0.844349] pps_core: LinuxPPS API ver. 1 registered [ 0.849013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.857061] PTP clock support registered [ 0.862084] EDAC MC: Ver: 3.0.0 [ 0.867152] PCI: Using ACPI for IRQ routing [ 0.871545] NetLabel: Initializing [ 0.873013] NetLabel: domain hash size = 128 [ 0.877013] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.880115] NetLabel: unlabeled traffic allowed by default [ 0.883348] vgaarb: loaded [ 0.885231] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.888016] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.899427] clocksource: Switched to clocksource kvm-clock [ 1.178137] VFS: Disk quotas dquot_6.6.0 [ 1.180974] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.187270] *** VALIDATE ramfs *** [ 1.190279] *** VALIDATE hugetlbfs *** [ 1.194025] pnp: PnP ACPI init [ 1.200303] pnp: PnP ACPI: found 6 devices [ 1.224416] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.229901] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.232716] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.236933] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.241953] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.247846] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.254218] NET: Registered protocol family 2 [ 1.256521] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.269518] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.274821] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.286812] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.294909] TCP: Hash tables configured (established 65536 bind 65536) [ 1.301648] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.308649] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.316500] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.320379] NET: Registered protocol family 1 [ 1.326059] RPC: Registered named UNIX socket transport module. [ 1.331645] RPC: Registered udp transport module. [ 1.335664] RPC: Registered tcp transport module. [ 1.338971] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.345388] NET: Registered protocol family 44 [ 1.348544] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.354207] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.358291] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.363256] PCI: CLS 0 bytes, default 64 [ 1.366414] Unpacking initramfs... [ 4.321521] debug: unmapping init [mem 0xffff988dfcc54000-0xffff988dfffbffff] [ 4.333183] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 4.336418] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 4.339846] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 7.153444] Initialise system trusted keyrings [ 7.155623] Key type blacklist registered [ 7.157669] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 7.170573] zbud: loaded [ 7.173809] *** VALIDATE nfs *** [ 7.175032] *** VALIDATE nfs4 *** [ 7.176752] pstore: using deflate compression [ 7.181161] Platform Keyring initialized [ 7.356618] NET: Registered protocol family 38 [ 7.361650] Key type asymmetric registered [ 7.365886] Asymmetric key parser 'x509' registered [ 7.372646] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 7.384045] io scheduler mq-deadline registered [ 7.389950] io scheduler kyber registered [ 7.395839] io scheduler bfq registered [ 7.400762] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 7.410787] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 7.417219] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 7.421277] ACPI: Power Button [PWRF] [ 7.428733] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 7.441217] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 7.478925] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 7.518423] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 7.573891] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 7.657396] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 7.720904] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 7.740642] Non-volatile memory driver v1.3 [ 7.746890] Linux agpgart interface v0.103 [ 7.855273] virtio_blk virtio1: [vda] 149376 512-byte logical blocks (76.5 MB/72.9 MiB) [ 7.863821] vda: detected capacity change from 0 to 76480512 [ 7.918738] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 7.924888] vdb: detected capacity change from 0 to 1073741824 [ 7.968724] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 7.977960] vdc: detected capacity change from 0 to 2621440000 [ 8.012702] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 8.024298] vdd: detected capacity change from 0 to 2621440000 [ 8.056902] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 8.060523] vde: detected capacity change from 0 to 4294967296 [ 8.080192] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 8.083436] vdf: detected capacity change from 0 to 4294967296 [ 8.090392] libphy: Fixed MDIO Bus: probed [ 8.097279] usbcore: registered new interface driver usbserial_generic [ 8.100391] usbserial: USB Serial support registered for generic [ 8.103482] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 8.110630] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 8.113132] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 8.116402] mousedev: PS/2 mouse device common for all mice [ 8.120484] rtc_cmos 00:05: RTC can wake from S4 [ 8.123966] rtc_cmos 00:05: registered as rtc0 [ 8.126510] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 8.129909] intel_pstate: CPU model not supported [ 8.134441] hid: raw HID events driver (C) Jiri Kosina [ 8.137580] usbcore: registered new interface driver usbhid [ 8.141227] usbhid: USB HID core driver [ 8.142744] drop_monitor: Initializing network drop monitor service [ 8.150149] Initializing XFRM netlink socket [ 8.154505] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 8.161089] NET: Registered protocol family 10 [ 8.175357] Segment Routing with IPv6 [ 8.177622] NET: Registered protocol family 17 [ 8.181302] mpls_gso: MPLS GSO support [ 8.182387] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 8.203344] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 8.216237] RAS: Correctable Errors collector initialized. [ 8.220507] AVX version of gcm_enc/dec engaged. [ 8.225286] AES CTR mode by8 optimization enabled [ 8.410902] sched_clock: Marking stable (8410733427, 0)->(9855457616, -1444724189) [ 8.415644] registered taskstats version 1 [ 8.422699] Loading compiled-in X.509 certificates [ 8.428548] zswap: loaded using pool lzo/zbud [ 8.501599] Key type big_key registered [ 8.535873] Key type encrypted registered [ 8.541545] ima: No TPM chip found, activating TPM-bypass! [ 8.546383] ima: Allocated hash algorithm: sha1 [ 8.549524] ima: No architecture policies found [ 8.553278] evm: Initialising EVM extended attributes: [ 8.555563] evm: security.selinux [ 8.557873] evm: security.ima [ 8.560489] evm: security.capability [ 8.563338] evm: HMAC attrs: 0x1 [ 8.568281] rtc_cmos 00:05: setting system clock to 2026-08-10 22:13:55 UTC (1786400035) [ 8.578909] debug: unmapping init [mem 0xffffffffa6a03000-0xffffffffa6bfffff] [ 8.584572] debug: unmapping init [mem 0xffffffffa5782000-0xffffffffa5a58fff] [ 8.599499] Write protecting the kernel read-only data: 28672k [ 8.608068] debug: unmapping init [mem 0xffffffffa3e03000-0xffffffffa3ffffff] [ 8.612528] debug: unmapping init [mem 0xffffffffa4714000-0xffffffffa47fffff] [ 8.681330] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 8.695215] systemd[1]: Detected virtualization kvm. [ 8.697626] systemd[1]: Detected architecture x86-64. [ 8.701216] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 8.739350] systemd[1]: No hostname configured. [ 8.744301] systemd[1]: Set hostname to . [ 8.749815] random: systemd: uninitialized urandom read (16 bytes read) [ 8.757315] systemd[1]: Initializing machine ID from random generator. [ 8.857944] random: ln: uninitialized urandom read (6 bytes read) [ 9.089464] random: systemd: uninitialized urandom read (16 bytes read) [ 9.093181] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 9.105492] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 9.115495] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ OK ] Listening on udev Control Socket. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Setup Virtual Console... [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket (/dev/log). Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Initrd Root Device. Starting Journal Service... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. Starting Apply Kernel Variables... [ OK ] Reached target Sockets. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 10.756917] device-mapper: uevent: version 1.0.3 [ 10.759772] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 12.358336] virtio_net virtio0 ens2: renamed from eth0 [ 12.628171] random: fast init done [ 12.727540] scsi host0: ata_piix [ 12.774700] scsi host1: ata_piix [ 12.782372] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 12.801441] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 18.082735] random: crng init done [ 18.088665] random: 7 urandom warning(s) missed due to ratelimiting [ 20.451154] dracut-initqueue[583]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 22.884315] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 25.991622] printk: systemd: 26 output lines suppressed due to ratelimiting [ 26.737900] SELinux: Disabled at runtime. [ 26.833994] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 26.849948] systemd[1]: Detected virtualization kvm. [ 26.853767] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 28.883884] systemd[1]: initrd-switch-root.service: Succeeded. [ 28.901355] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 28.917237] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 28.932419] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 28.951632] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 28.989924] systemd[1]: Starting Journal Service... Starting Journal Service... [ 29.019691] systemd[1]: Mounting Huge Pages File System... Mounting Huge Pages File System... [ OK ] Listening on udev Control Socket. [ OK ] Created slice system-getty.slice. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ 29.233948] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... Starting Apply Kernel Variables... Starting Remount Root and Kernel File Systems... Mounting Kernel Debug File System... [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-sshd\x2dkeygen.slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Mounting POSIX Message Queue File System... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started udev Coldplug all Devices. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 31.407997] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 32.963977] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 33.282937] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 33.587341] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 33.871115] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (9s / no limit) [** ] A start job is running for Configur…-only root support (9s / no limit) [*** ] A start job is running for Configur…-only root support (9s / no limit)[ 39.146913] Key type dns_resolver registered [ *** ] A start job is running for Configur…only root support (10s / no limit) [ *** ] A start job is running for Configur…only root support (11s / no limit) [ ***] A start job is running for Configur…only root support (11s / no limit)[ 40.321518] NFS: Registering the id_resolver key type [ 40.328782] Key type id_resolver registered [ 40.336652] Key type id_legacy registered [ **] A start job is running for Configur…only root support (12s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started OpenSSH server daemon. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg435-server login: [ 96.967446] libcfs: loading out-of-tree module taints kernel. [ 97.076418] Key type ._llcrypt registered [ 97.078522] Key type .llcrypt registered [ 97.183847] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_hostid [ 116.671691] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 118.354184] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 118.370331] alg: No test for adler32 (adler32-zlib) [ 119.855905] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [ 120.484282] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [ 122.351251] Key type lgssc registered [ 124.139260] Lustre: Echo OBD driver; http://www.lustre.org/ [ 131.856123] hrtimer: interrupt took 2521070 ns [ 140.871041] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 183.338755] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 195.942353] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 195.989928] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 197.235906] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 197.286425] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 197.389748] Lustre: lustre-MDT0000: new disk, initializing [ 197.452624] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 197.466401] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 201.354814] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 215.273941] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 215.435290] Lustre: 6509:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 215.493167] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 215.498926] Lustre: Skipped 1 previous similar message [ 215.646431] Lustre: lustre-MDT0001: new disk, initializing [ 215.753309] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 215.807883] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 215.841516] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 220.755834] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 225.782022] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 234.910775] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 235.238755] Lustre: lustre-OST0000: new disk, initializing [ 235.242912] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 235.248315] Lustre: 8447:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 235.318133] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 240.191160] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 240.200230] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 240.374492] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 241.653320] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 256.409174] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 256.549818] Lustre: lustre-OST0001: new disk, initializing [ 256.553749] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 256.559637] Lustre: 9519:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 256.636227] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 262.547394] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 266.328602] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 266.352500] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 266.563127] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 276.943349] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 285.801638] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 292.545970] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing check_logdir /tmp/testlogs/ [ 298.325573] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing yml_node [ 303.773442] Lustre: DEBUG MARKER: Client: 2.17.54.107 [ 306.925943] Lustre: DEBUG MARKER: MDS: 2.17.54.107 [ 310.349417] Lustre: DEBUG MARKER: OSS: 2.17.54.107 [ 313.174425] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Aug 10 18:18:58 EDT 2026 [ 331.908733] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 333.715569] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 348.644344] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 348.649327] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 348.663315] Lustre: Skipped 3 previous similar messages [ 348.668209] Lustre: Skipped 2 previous similar messages [ 353.760382] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 353.771276] Lustre: Skipped 3 previous similar messages [ 354.823937] Lustre: server umount lustre-MDT0000 complete [ 358.356211] LustreError: 6500:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786400385 with bad export cookie 13731458053907324928 [ 358.363694] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 358.370816] LustreError: 6500:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 358.886217] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 358.888721] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 358.897147] Lustre: Skipped 1 previous similar message [ 358.912363] Lustre: Skipped 1 previous similar message [ 364.000793] LustreError: 7891:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 364.000869] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 364.020052] LustreError: 7891:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 364.681912] Lustre: server umount lustre-MDT0001 complete [ 367.888366] Lustre: server umount lustre-OST0000 complete [ 370.893205] Lustre: server umount lustre-OST0001 complete [ 376.779207] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_hostid [ 385.513943] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 420.441183] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 420.630154] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 420.679724] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 420.777643] Lustre: lustre-MDT0000: new disk, initializing [ 420.838822] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 420.855591] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 425.228120] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 435.226622] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 435.364970] Lustre: 15072:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 435.431877] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 435.436279] Lustre: Skipped 1 previous similar message [ 435.524244] Lustre: lustre-MDT0001: new disk, initializing [ 435.683423] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 435.753163] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 435.761776] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 440.745467] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 451.635878] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 458.109165] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 466.050190] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 466.456406] Lustre: lustre-OST0000: new disk, initializing [ 466.467614] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 466.478527] Lustre: 17040:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 466.613694] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 467.912051] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 467.934839] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 468.099449] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 473.190536] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 482.021326] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 486.164941] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 486.447618] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 490.901354] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 491.160944] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 493.542089] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 493.550449] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 493.577084] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 493.595848] Lustre: Skipped 2 previous similar messages [ 499.412727] Lustre: server umount lustre-OST0000 complete [ 503.784147] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 503.800760] Lustre: Skipped 1 previous similar message [ 503.808045] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 503.814849] Lustre: Skipped 3 previous similar messages [ 518.111862] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 518.370249] Lustre: server umount lustre-MDT0000 complete [ 519.140829] LustreError: 15079:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 519.158247] LustreError: 15079:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 521.820294] LustreError: 15063:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786400548 with bad export cookie 13731458053907326461 [ 521.823413] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 521.833642] LustreError: 15063:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 522.152681] Lustre: server umount lustre-MDT0001 complete [ 526.801996] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 18:22:33 (1786400553) [ 533.946566] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 534.322088] LustreError: 19307:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 534.407248] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 539.623799] LustreError: 19308:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 540.040675] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 544.739174] LustreError: 19307:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 549.107650] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 555.019957] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 564.008398] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 570.935892] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 578.794727] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 586.591921] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 594.845970] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 598.183805] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 605.164860] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 605.171274] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 605.196505] Lustre: Skipped 3 previous similar messages [ 605.219282] Lustre: Skipped 6 previous similar messages [ 609.774523] Lustre: server umount lustre-MDT0000 complete [ 610.280219] LustreError: 19302:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 610.315875] LustreError: 19302:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 614.290269] LustreError: 19287:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786400641 with bad export cookie 13731458053907327343 [ 614.299572] LustreError: 19287:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 614.306035] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 614.651934] Lustre: server umount lustre-MDT0001 complete [ 629.549595] Lustre: server umount lustre-OST0000 complete [ 651.751070] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 661.271714] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 661.518899] LustreError: 23692:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 661.634784] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 661.647355] Lustre: Skipped 2 previous similar messages [ 666.164092] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 674.140000] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 678.799860] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 681.445747] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 689.277466] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 697.860231] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 706.999264] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 707.208450] Lustre: lustre-OST0001: new disk, initializing [ 707.212818] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 707.219523] Lustre: 26009:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 708.484965] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 708.520023] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 708.723099] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 713.119591] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 722.940355] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 727.081235] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 741.346467] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 741.354517] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 741.372792] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 745.869047] Lustre: server umount lustre-MDT0000 complete [ 749.558170] LustreError: 23692:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 749.596470] LustreError: 23692:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 749.852487] LustreError: 23674:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786400776 with bad export cookie 13731458053907328190 [ 749.853686] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 749.874375] LustreError: 23674:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 750.188640] Lustre: server umount lustre-MDT0001 complete [ 764.996789] Lustre: server umount lustre-OST0000 complete [ 778.450286] Lustre: server umount lustre-OST0001 complete [ 794.047538] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 803.099868] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 803.553241] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 803.560398] Lustre: Skipped 3 previous similar messages [ 807.644813] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 814.055373] LustreError: 29027:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 814.097439] LustreError: 29027:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 816.062780] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 820.382375] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 823.240487] Lustre: 30172:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 829.409540] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 834.855632] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 843.900535] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 850.310881] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 857.102410] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 860.774767] Lustre: 32049:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 867.809458] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 867.821431] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 867.836962] Lustre: Skipped 3 previous similar messages [ 867.845878] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 867.853504] Lustre: Skipped 4 previous similar messages [ 871.394408] Lustre: server umount lustre-MDT0000 complete [ 874.631070] LustreError: 32051:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786400901 with bad export cookie 13731458053907329485 [ 874.635085] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 874.637789] LustreError: 32051:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 874.880971] Lustre: server umount lustre-MDT0001 complete [ 888.371213] Lustre: server umount lustre-OST0000 complete [ 901.542296] Lustre: server umount lustre-OST0001 complete [ 909.826468] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 18:28:56 (1786400936) [ 938.154360] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 950.837499] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 951.326443] LustreError: 34854:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 951.346246] LustreError: 34854:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 955.862812] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 964.421224] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 969.206629] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 971.952292] Lustre: 36004:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 978.295196] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 985.496331] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 993.924787] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1000.769641] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 1009.859804] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1014.316706] Lustre: 37886:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1020.896989] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1020.906746] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1020.925117] Lustre: Skipped 3 previous similar messages [ 1020.935429] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1020.943408] Lustre: Skipped 4 previous similar messages [ 1025.261888] Lustre: server umount lustre-MDT0000 complete [ 1030.232534] LustreError: 34834:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786401057 with bad export cookie 13731458053907330591 [ 1030.246393] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1030.253868] LustreError: 34834:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1030.507704] Lustre: server umount lustre-MDT0001 complete [ 1044.808759] Lustre: server umount lustre-OST0000 complete [ 1048.358378] Lustre: 3642:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786401059/real 1786401059] req@ffff988e6f724000 x1873176320879872/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786401075 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1048.958982] Lustre: server umount lustre-OST0001 complete [ 1058.245273] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 18:31:25 (1786401085) [ 1073.572131] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 1084.977176] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1085.512284] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1085.517979] Lustre: Skipped 7 previous similar messages [ 1090.362146] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 1099.548968] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1104.293312] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 1107.075974] Lustre: 41177:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1113.055237] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1119.608786] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 1128.697588] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1136.024997] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 1144.341987] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1148.082850] Lustre: 43051:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1148.888328] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1148.892773] Lustre: Skipped 1 previous similar message [ 1156.066934] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1156.073400] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1156.091342] Lustre: Skipped 4 previous similar messages [ 1158.095623] Lustre: server umount lustre-MDT0000 complete [ 1162.110807] LustreError: 40008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786401189 with bad export cookie 13731458053907331697 [ 1162.120869] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1162.130751] LustreError: 40008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1168.538717] Lustre: server umount lustre-MDT0001 complete [ 1178.035826] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1187.299916] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0xbe8ff058ff52bed8 [ 1187.316078] Lustre: MGC192.168.204.135@tcp: Connection restored to 0@lo (at 0@lo) [ 1193.029752] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 1197.159566] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1202.206691] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1207.379802] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 1207.819339] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1207.826627] Lustre: Skipped 1 previous similar message [ 1215.475950] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1217.405396] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1224.718098] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1227.801580] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1239.383167] Lustre: server umount lustre-MDT0000 complete [ 1243.624346] LustreError: 43928:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1243.654935] LustreError: 43928:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 35 previous similar messages [ 1244.121759] LustreError: 40778:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786401271 with bad export cookie 13731458053907332824 [ 1244.127743] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1244.132968] LustreError: 40778:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1244.491067] Lustre: server umount lustre-MDT0001 complete [ 1259.086764] Lustre: server umount lustre-OST0000 complete [ 1272.610365] Lustre: server umount lustre-OST0001 complete [ 1280.446729] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 18:35:07 (1786401307) [ 1296.508838] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 1306.639521] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1311.501677] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 1320.018610] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1326.129336] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 1329.464605] Lustre: 48815:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1336.122292] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1342.401488] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 1350.391526] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1356.742747] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 1364.059616] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1369.025639] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1381.187323] Lustre: server umount lustre-MDT0000 complete [ 1381.864909] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1381.876609] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1381.902610] Lustre: Skipped 9 previous similar messages [ 1385.204390] LustreError: 50814:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786401412 with bad export cookie 13731458053907333622 [ 1385.221701] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1385.223852] LustreError: 50814:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1385.648174] Lustre: server umount lustre-MDT0001 complete [ 1399.857619] Lustre: server umount lustre-OST0000 complete [ 1414.117506] Lustre: server umount lustre-OST0001 complete [ 1424.252821] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 18:37:30 (1786401450) [ 1426.899761] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1428.836058] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 18:37:35 (1786401455) [ 1439.846865] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_hostid [ 1449.893648] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 1477.531109] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1477.991670] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1478.137302] Lustre: lustre-MDT0000: new disk, initializing [ 1478.261267] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1483.594155] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 1493.527984] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1499.922816] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1500.179975] Lustre: lustre-OST0000: new disk, initializing [ 1500.186035] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1500.192546] Lustre: Skipped 1 previous similar message [ 1500.199949] Lustre: 54894:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1501.401099] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1501.408952] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1501.431812] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1506.930746] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 1516.667368] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1519.575058] Lustre: DEBUG MARKER: create test files [ 1547.747054] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1547.757675] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1547.761383] Lustre: Skipped 22 previous similar messages [ 1551.502804] Lustre: server umount lustre-OST0000 complete [ 1555.669741] Lustre: server umount lustre-MDT0000 complete [ 1566.351742] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [ 1569.614146] Key type lgssc unregistered [ 1570.075629] LNet: 56526:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1570.088522] LNetError: 56526:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1571.109851] LNet: Removed LNI 192.168.204.135@tcp [ 1572.077344] Key type .llcrypt unregistered [ 1572.081552] Key type ._llcrypt unregistered [ 1604.946978] Key type ._llcrypt registered [ 1604.951083] Key type .llcrypt registered [ 1605.110522] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 1606.286986] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1606.308947] alg: No test for adler32 (adler32-zlib) [ 1607.388952] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [ 1607.809979] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [ 1609.472078] Key type lgssc registered [ 1611.004573] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1619.979706] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1620.024171] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1621.465346] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1626.562968] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 1634.254978] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1640.594758] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1640.958255] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1647.101228] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1647.183245] Lustre: 59262:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1647.201207] Lustre: 59262:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1647.213306] Lustre: 59262:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1647.231859] Lustre: 59262:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1647.257046] Lustre: 59262:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1647.264996] Lustre: 59262:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1648.539839] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 1657.145296] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1664.188852] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1670.267718] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1677.796491] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1677.806947] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1677.810825] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1682.914609] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1683.148626] Lustre: server umount lustre-OST0000 complete [ 1687.612587] Lustre: server umount lustre-MDT0000 complete [ 1699.685688] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [ 1702.981232] Key type lgssc unregistered [ 1703.363691] LNet: 60873:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1703.387587] LNetError: 60873:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1704.423976] LNet: Removed LNI 192.168.204.135@tcp [ 1705.479330] Key type .llcrypt unregistered [ 1705.481176] Key type ._llcrypt unregistered [ 1741.754560] Key type ._llcrypt registered [ 1741.763495] Key type .llcrypt registered [ 1742.022934] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 1743.623381] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1743.661682] alg: No test for adler32 (adler32-zlib) [ 1744.776669] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [ 1745.064951] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [ 1746.727477] Key type lgssc registered [ 1748.814797] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1758.813829] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1758.837876] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1760.305557] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1765.263368] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 1772.813813] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1779.717292] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1780.014794] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1785.156491] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1785.249217] Lustre: 63712:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1785.265794] Lustre: 63712:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1785.272659] Lustre: 63712:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1785.285061] Lustre: 63712:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1785.294948] Lustre: 63712:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1785.305915] Lustre: 63712:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1786.887647] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 1795.673476] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1801.152898] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1810.914214] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1810.929274] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1816.037376] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1816.624942] Lustre: server umount lustre-OST0000 complete [ 1820.723819] Lustre: server umount lustre-MDT0000 complete [ 1833.560716] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [ 1836.616348] Key type lgssc unregistered [ 1836.952266] LNet: 65268:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1836.957317] LNetError: 65268:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1836.966780] LNet: Removed LNI 192.168.204.135@tcp [ 1837.995464] Key type .llcrypt unregistered [ 1837.998961] Key type ._llcrypt unregistered [ 1858.610932] Key type ._llcrypt registered [ 1858.613798] Key type .llcrypt registered [ 1858.803888] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_hostid [ 1878.230485] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 1879.944776] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1879.978463] alg: No test for adler32 (adler32-zlib) [ 1881.212949] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [ 1881.630108] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [ 1883.367193] Key type lgssc registered [ 1884.841547] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1925.926809] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1925.960378] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1927.139762] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1927.162243] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1927.230631] Lustre: lustre-MDT0000: new disk, initializing [ 1927.286307] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1927.316220] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1932.281616] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 1945.465182] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1945.635384] Lustre: 68424:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1945.704418] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1945.709110] Lustre: Skipped 1 previous similar message [ 1945.781660] Lustre: lustre-MDT0001: new disk, initializing [ 1945.945271] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1946.040191] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1946.065584] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1951.547280] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 1962.700763] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1971.383267] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1979.705031] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1980.111995] Lustre: lustre-OST0000: new disk, initializing [ 1980.118719] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1980.124816] Lustre: 70397:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1980.284206] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1987.683507] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1987.694597] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1987.848248] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1987.915602] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 1999.506830] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2005.403913] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2005.803814] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2011.467721] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2011.716747] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2018.791056] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2018.791901] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2018.804390] LustreError: Skipped 1 previous similar message [ 2018.812850] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2018.816815] Lustre: Skipped 1 previous similar message [ 2020.394968] Lustre: server umount lustre-OST0000 complete [ 2028.000808] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2028.014046] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2028.038751] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2028.049534] Lustre: Skipped 1 previous similar message [ 2029.026928] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2029.044135] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2029.051712] Lustre: Skipped 1 previous similar message [ 2030.911798] Lustre: server umount lustre-MDT0000 complete [ 2035.545243] LustreError: 68417:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786402062 with bad export cookie 1393252231467731329 [ 2035.558129] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2035.980379] Lustre: server umount lustre-MDT0001 complete [ 2048.916774] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 18:47:54 (1786402074) [ 2064.432546] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 2109.125646] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_hostid [ 2119.278854] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 2157.506390] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2157.768977] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2157.796106] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2157.858983] Lustre: lustre-MDT0000: new disk, initializing [ 2157.932172] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2157.953635] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2163.138904] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 2174.741674] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2174.942863] Lustre: 76431:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2175.005633] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2175.010713] Lustre: Skipped 1 previous similar message [ 2175.100897] Lustre: lustre-MDT0001: new disk, initializing [ 2175.227818] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2175.283362] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2175.304949] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2180.421534] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 2191.520689] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2197.946903] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2204.993193] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2205.398873] Lustre: lustre-OST0000: new disk, initializing [ 2205.402492] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2205.410503] Lustre: 78398:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2205.504115] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2210.857384] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2210.867110] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2211.016100] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2211.875417] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 2221.168728] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2226.973253] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2227.205558] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2233.361719] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2233.752211] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2236.904927] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2236.909232] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2236.932531] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2236.937032] LustreError: Skipped 1 previous similar message [ 2236.963626] Lustre: Skipped 1 previous similar message [ 2241.913879] Lustre: server umount lustre-OST0000 complete [ 2247.147591] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2247.157509] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2247.162241] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2247.163419] Lustre: Skipped 2 previous similar messages [ 2252.066583] Lustre: server umount lustre-MDT0000 complete [ 2252.257859] LustreError: 76441:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2252.283168] LustreError: 76441:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 2256.235988] LustreError: 76423:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786402283 with bad export cookie 1393252231467732211 [ 2256.256985] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2256.257671] LustreError: 76423:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 2256.694919] Lustre: server umount lustre-MDT0001 complete [ 2265.883995] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 18:51:32 (1786402292) [ 2272.248974] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2272.602593] LustreError: 80813:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2272.688996] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2277.859424] LustreError: 80814:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2278.627986] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 2282.986292] LustreError: 80813:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2287.952811] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2288.098541] LustreError: 80814:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2294.497066] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 2302.675576] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2308.173851] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2316.257439] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2316.521478] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2316.527161] Lustre: Skipped 1 previous similar message [ 2324.019827] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 2332.314361] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2336.441560] Lustre: 83486:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting ff6eabdb-a3ae-45e6-8399-e73eb7131a84 at adminstrative request [ 2339.304260] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2339.326121] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2349.556671] Lustre: MGS: Client ff6eabdb-a3ae-45e6-8399-e73eb7131a84 (at 0@lo) reconnecting [ 2349.561921] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x1355d385d3279065 to 0x1355d385d3279305 [ 2349.579659] Lustre: MGC192.168.204.135@tcp: Connection restored to 0@lo (at 0@lo) [ 2373.871141] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2374.071461] Lustre: lustre-OST0001: new disk, initializing [ 2374.074719] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2374.080460] Lustre: 83880:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2380.929056] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2380.956973] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2381.101908] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2381.294792] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 2391.522709] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2396.643205] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2396.647117] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2396.651355] LustreError: Skipped 1 previous similar message [ 2396.657680] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2396.669165] Lustre: Skipped 1 previous similar message [ 2396.683710] Lustre: Skipped 2 previous similar messages [ 2401.086043] Lustre: server umount lustre-OST0001 complete [ 2401.801232] LustreError: 84217:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2401.850258] LustreError: 84217:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2412.689850] Lustre: server umount lustre-OST0000 complete [ 2420.193078] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2420.198886] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2420.209653] Lustre: Skipped 3 previous similar messages [ 2420.228891] Lustre: Skipped 5 previous similar messages [ 2423.585646] Lustre: server umount lustre-MDT0000 complete [ 2425.314153] LustreError: 82115:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2425.342101] LustreError: 82115:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 2427.696010] LustreError: 80795:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786402454 with bad export cookie 1393252231467733765 [ 2427.703176] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2427.707410] LustreError: 80795:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2428.202626] Lustre: server umount lustre-MDT0001 complete [ 2437.112245] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 18:54:23 (1786402463) [ 2438.523515] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2440.508593] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 18:54:27 (1786402467) [ 2442.166281] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2444.570444] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 18:54:30 (1786402470) [ 2446.105862] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2447.746101] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 18:54:34 (1786402474) [ 2449.338013] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2451.132840] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 18:54:37 (1786402477) [ 2452.879697] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2458.991029] loop: module loaded [ 2459.019660] loop0: detected capacity change from 0 to 4294967296 [ 2459.094232] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2461.314982] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2465.860560] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2469.921419] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2499.818383] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2499.867258] LustreError: 88005:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2499.874172] LustreError: 88005:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2499.878847] LustreError: 88005:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2499.884788] LustreError: 88005:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2521.434141] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_hostid [ 2531.744532] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 2573.233073] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2573.657615] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2573.701538] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2573.818984] Lustre: lustre-MDT0000: new disk, initializing [ 2573.984326] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2573.995145] Lustre: Skipped 1 previous similar message [ 2574.031671] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2579.128731] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 2591.334294] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2591.652344] Lustre: 90872:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2591.941473] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2597.130416] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 2607.846282] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2614.864598] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2623.092890] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2623.326704] Lustre: 92841:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2630.975222] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 2631.343084] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2631.581536] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2641.774910] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2646.671251] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2647.010617] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2652.000909] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2652.387530] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2657.248686] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2657.261453] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2657.273186] LustreError: Skipped 1 previous similar message [ 2657.275788] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2657.306368] Lustre: Skipped 1 previous similar message [ 2661.205780] Lustre: server umount lustre-OST0000 complete [ 2671.989533] Lustre: server umount lustre-MDT0000 complete [ 2672.611936] LustreError: 94213:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2672.634516] LustreError: 94213:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2676.930557] LustreError: 90865:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786402703 with bad export cookie 1393252231467734381 [ 2676.937270] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2676.956721] LustreError: 90865:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2677.421749] Lustre: server umount lustre-MDT0001 complete [ 2688.045121] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 18:58:34 (1786402714) [ 2697.283466] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2703.532379] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 2713.654952] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2714.005822] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2714.013478] Lustre: Skipped 3 previous similar messages [ 2718.694207] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 2727.485967] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2733.615594] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2741.039277] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2748.545790] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 2756.897317] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2761.400944] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2761.650314] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2766.646161] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2766.940255] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2774.681539] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2775.075358] Lustre: lustre-OST0001: new disk, initializing [ 2775.081479] Lustre: Skipped 2 previous similar messages [ 2775.086639] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2775.096341] Lustre: Skipped 3 previous similar messages [ 2775.108682] Lustre: 98625:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2783.872478] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 2784.929205] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2784.944309] Lustre: Skipped 2 previous similar messages [ 2784.957125] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2785.135947] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2793.900707] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2800.308861] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2800.516570] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2806.361679] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2806.630231] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2828.435766] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2856.224322] Lustre: Failing over lustre-MDT0000 [ 2856.680685] Lustre: server umount lustre-MDT0000 complete [ 2856.944686] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2856.957382] LustreError: 95250:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2856.979956] Lustre: Skipped 5 previous similar messages [ 2857.020596] LustreError: 95250:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [ 2877.408493] Lustre: 66649:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786402888/real 1786402888] req@0000000098b4a63d x1873178167370240/t0(0) o400->MGC192.168.204.135@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1786402904 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2877.443838] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2886.309480] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2886.315758] LDISKFS-fs (dm-0): recovery complete [ 2886.325048] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2887.519842] LustreError: 101460:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 2887.528636] LustreError: 101460:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@000000002adb1e0c x1873178167375488/t0(0) o250->MGC192.168.204.135@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1786402914 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2887.555424] LustreError: 101460:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 2887.657585] LustreError: 66646:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000f69b0c9b x1873178167376768/t0(0) o250->MGC192.168.204.135@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2888.158955] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2891.257893] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2891.257936] Lustre: 97793:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2893.289204] Lustre: 95985:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2893.334026] Lustre: 95985:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 1 previous similar message [ 2893.364407] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2893.449081] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2893.767120] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2894.136397] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2894.155307] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 2894.497414] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2894.506256] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 2894.527820] Lustre: 101491:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2894.547541] Lustre: 101491:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 7 previous similar messages [ 2894.615762] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 2895.245781] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2895.261551] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 2 previous similar messages [ 2895.599205] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2895.617572] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 2 previous similar messages [ 2896.678090] Lustre: 101491:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2896.719707] Lustre: 101491:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 5 previous similar messages [ 2897.427972] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2897.453252] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 5 previous similar messages [ 2897.792281] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2897.804347] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 5 previous similar messages [ 2900.697561] Lustre: 101491:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2900.710508] Lustre: 101491:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 11 previous similar messages [ 2901.677984] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2901.688881] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 2902.015156] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2902.022463] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 2909.039246] Lustre: 101491:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2909.060417] Lustre: 101491:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 25 previous similar messages [ 2909.744661] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2909.769647] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 24 previous similar messages [ 2910.095163] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2910.102571] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 24 previous similar messages [ 2925.257463] Lustre: 101491:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2925.277496] Lustre: 101491:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 50 previous similar messages [ 2925.922488] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2925.950775] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 50 previous similar messages [ 2926.271311] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2926.279023] LustreError: 101491:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 50 previous similar messages [ 2951.047234] Lustre: 101491:0:(ldlm_lib.c:2380:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2951.055136] Lustre: 101491:0:(ldlm_lib.c:2390:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2951.072416] Lustre: 101491:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client b468a34c-7219-42b8-8efe-01d6e4c1e28f@192.168.204.35@tcp [ 2951.095403] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2951.105350] Lustre: 101491:0:(ldlm_lib.c:1900:abort_req_replay_queue()) @@@ aborted: req@0000000010b7cc18 x1873178148279296/t0(8589934776) o35->b468a34c-7219-42b8-8efe-01d6e4c1e28f@192.168.204.35@tcp:508/0 lens 392/0 e 0 to 0 dl 1786402988 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2951.170177] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2951.185822] Lustre: lustre-MDT0000: Denying connection for new client b468a34c-7219-42b8-8efe-01d6e4c1e28f (at 192.168.204.35@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2951.258328] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2951.354885] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2951.361087] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2951.373755] Lustre: Skipped 2 previous similar messages [ 2951.487854] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 2951.488414] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 2970.105906] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2970.125288] Lustre: Skipped 4 previous similar messages [ 2975.887277] Lustre: server umount lustre-OST0000 complete [ 2983.394087] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2985.454221] LustreError: 97078:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2985.489391] LustreError: 97078:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 39 previous similar messages [ 2995.169910] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2995.434762] Lustre: server umount lustre-OST0001 complete [ 3002.849378] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3002.865877] Lustre: Skipped 4 previous similar messages [ 3007.702199] Lustre: server umount lustre-MDT0000 complete [ 3012.835484] LustreError: 95235:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786403039 with bad export cookie 1393252231467785593 [ 3012.838653] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3012.846624] LustreError: 95235:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3013.496813] Lustre: server umount lustre-MDT0001 complete [ 3026.119516] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 19:04:11 (1786403051) [ 3034.565420] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3034.571428] Lustre: *** cfs_fail_loc=197, val=0*** [ 3050.143820] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3055.263476] LustreError: 103467:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.135@tcp: failed processing log, type 4: rc = -110 [ 3060.422730] Lustre: *** cfs_fail_loc=197, val=0*** [ 3060.425547] Lustre: Skipped 65 previous similar messages [ 3080.992576] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3081.005554] Lustre: Skipped 3 previous similar messages [ 3090.141200] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 3100.643521] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3103.583222] Lustre: server umount lustre-OST0000 complete [ 3116.007568] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 19:05:41 (1786403141) [ 3145.184854] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3153.335752] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 3167.160981] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3172.150056] LustreError: 105119:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3172.160746] LustreError: 105119:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3172.168439] LustreError: 105119:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 26, retries 0, failed: rc = -5 [ 3172.495878] Lustre: server umount lustre-MDT0000 complete [ 3217.281436] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_hostid [ 3233.911577] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 3283.653863] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3284.129699] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3284.192897] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3284.341846] Lustre: lustre-MDT0000: new disk, initializing [ 3284.513241] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3292.164685] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 3309.123953] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3309.417490] Lustre: 109321:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3309.827683] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3317.613659] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 3330.291495] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3337.687381] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3346.965713] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3347.296803] Lustre: 111291:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3355.456853] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3357.079556] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 3368.765904] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3374.293227] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3374.571271] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3379.869349] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3380.161353] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3386.336672] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3386.355570] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3386.362195] LustreError: Skipped 3 previous similar messages [ 3386.364266] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3386.364278] Lustre: Skipped 11 previous similar messages [ 3386.413356] Lustre: Skipped 2 previous similar messages [ 3388.743178] Lustre: server umount lustre-OST0000 complete [ 3408.351149] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3408.580637] Lustre: server umount lustre-MDT0000 complete [ 3411.941924] LustreError: 109333:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3411.984435] LustreError: 109333:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 11 previous similar messages [ 3413.105089] LustreError: 111290:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786403440 with bad export cookie 1393252231467796254 [ 3413.119055] LustreError: 111290:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3413.134974] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3413.469676] Lustre: server umount lustre-MDT0001 complete [ 3425.342619] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 19:10:51 (1786403451) [ 3438.514551] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [ 3441.453734] Key type lgssc unregistered [ 3441.889244] LNet: 113781:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3441.902658] LNetError: 113781:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3441.934809] LNet: Removed LNI 192.168.204.135@tcp [ 3443.335284] Key type .llcrypt unregistered [ 3443.337326] Key type ._llcrypt unregistered [ 3469.276756] Key type ._llcrypt registered [ 3469.290891] Key type .llcrypt registered [ 3469.433377] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_hostid [ 3496.395267] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 3499.042927] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3499.185588] alg: No test for adler32 (adler32-zlib) [ 3500.633201] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [ 3501.267384] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [ 3503.263236] Key type lgssc registered [ 3505.468494] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3555.198695] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3555.252621] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3556.545279] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3556.613463] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3556.774465] Lustre: lustre-MDT0000: new disk, initializing [ 3556.926422] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3556.977866] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3562.435563] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 3574.353144] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3574.648400] Lustre: lustre-OST0000: new disk, initializing [ 3574.652444] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3574.657145] Lustre: Skipped 1 previous similar message [ 3574.664190] Lustre: 117962:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3574.754209] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3583.012222] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 3584.230080] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3584.252741] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3584.322133] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3596.604199] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3596.777209] Lustre: lustre-OST0001: new disk, initializing [ 3596.781827] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3596.788591] Lustre: 118827:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3596.869749] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3603.643410] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3603.651732] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3603.706323] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3604.997741] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 3619.297147] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3619.306622] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3619.318663] Lustre: Skipped 1 previous similar message [ 3619.339633] Lustre: Skipped 1 previous similar message [ 3622.792838] Lustre: server umount lustre-MDT0000 complete [ 3631.135382] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3637.909861] LustreError: 117150:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786403664 with bad export cookie 12069061838845931432 [ 3637.910423] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3637.921166] LustreError: 117150:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3638.025845] Lustre: server umount lustre-OST0000 complete [ 3642.527541] Lustre: server umount lustre-OST0001 complete [ 3652.742397] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 19:14:39 (1786403679) [ 3680.815132] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 19:15:06 (1786403706) [ 3693.790553] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 3721.425174] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 19:15:47 (1786403747) [ 3764.066421] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_hostid [ 3777.208575] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 3819.827304] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3820.113241] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3820.153080] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3820.308510] Lustre: lustre-MDT0000: new disk, initializing [ 3820.442508] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3820.473532] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3826.157804] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 3837.198745] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3837.285302] Lustre: 126584:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3837.332585] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3837.336147] Lustre: Skipped 1 previous similar message [ 3837.403458] Lustre: lustre-MDT0001: new disk, initializing [ 3837.524873] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3837.559167] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3837.567686] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3841.683206] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 3852.383537] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3858.530681] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3865.315268] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3865.714351] Lustre: lustre-OST0000: new disk, initializing [ 3865.719187] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3865.729415] Lustre: 128553:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3865.964515] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3873.860547] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3873.891749] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3874.108223] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3874.636508] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 3886.308977] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3893.271909] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3893.780832] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3899.995269] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3900.203584] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3904.996222] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3905.014752] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3905.049058] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3905.077805] Lustre: Skipped 1 previous similar message [ 3908.734845] Lustre: server umount lustre-OST0000 complete [ 3914.208162] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3914.215959] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3914.231186] Lustre: Skipped 1 previous similar message [ 3914.237636] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3919.500955] Lustre: server umount lustre-MDT0000 complete [ 3923.390420] LustreError: 127378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786403950 with bad export cookie 12069061838845932657 [ 3923.401955] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3923.417790] LustreError: 127378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3923.721897] Lustre: server umount lustre-MDT0001 complete [ 3933.333740] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 19:19:20 (1786403960) [ 3940.133288] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3940.538701] LustreError: 130966:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3940.553083] LustreError: 130966:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3940.632058] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3945.953627] LustreError: 130967:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3946.065689] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 3951.076423] LustreError: 130966:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3956.237969] LustreError: 130967:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3956.931071] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3962.064810] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 3969.918638] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3976.546153] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3983.817259] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3984.116318] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3984.139116] Lustre: Skipped 1 previous similar message [ 3991.581831] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 4000.499639] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4008.765365] Lustre: *** cfs_fail_loc=159, val=0*** [ 4024.366331] Lustre: lustre-MDT0000: Client afff976a-aa34-4381-942b-d4d7587cdfd6 (at 192.168.204.35@tcp) reconnecting [ 4026.133841] Lustre: *** cfs_fail_loc=159, val=0*** [ 4026.145298] Lustre: Skipped 5 previous similar messages [ 4042.779540] Lustre: lustre-MDT0000: Client afff976a-aa34-4381-942b-d4d7587cdfd6 (at 192.168.204.35@tcp) reconnecting [ 4063.208236] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4063.217702] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4063.230764] Lustre: Skipped 1 previous similar message [ 4063.238898] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4063.254733] Lustre: Skipped 2 previous similar messages [ 4065.789324] Lustre: server umount lustre-OST0000 complete [ 4074.466894] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4074.469237] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4074.488366] Lustre: Skipped 2 previous similar messages [ 4074.498925] Lustre: Skipped 2 previous similar messages [ 4076.777920] Lustre: server umount lustre-MDT0000 complete [ 4079.586122] LustreError: 133576:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4079.621512] LustreError: 133576:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4081.330045] LustreError: 130947:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786404108 with bad export cookie 12069061838845933539 [ 4081.334824] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4081.354592] LustreError: 130947:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4081.691736] Lustre: server umount lustre-MDT0001 complete [ 4093.319629] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [ 4096.011675] Key type lgssc unregistered [ 4096.360436] LNet: 134755:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4096.374169] LNetError: 134755:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4096.397452] LNet: Removed LNI 192.168.204.135@tcp [ 4097.152834] Key type .llcrypt unregistered [ 4097.156275] Key type ._llcrypt unregistered [ 4122.229554] Key type ._llcrypt registered [ 4122.231843] Key type .llcrypt registered [ 4122.329762] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 4123.034197] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4123.050385] alg: No test for adler32 (adler32-zlib) [ 4124.160944] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [ 4124.409545] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [ 4126.079292] Key type lgssc registered [ 4127.529809] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4135.784415] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4135.803810] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4137.173887] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4141.460516] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 4147.692306] LustreError: 136169:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4147.712397] LustreError: 136169:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4150.109256] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4150.468501] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4155.095606] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 4162.585332] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4167.900888] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4174.707194] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4175.150670] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4182.197413] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 4182.335889] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4189.867343] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4200.498992] Lustre: *** cfs_fail_loc=159, val=0*** [ 4216.842932] Lustre: lustre-MDT0000: Client 5af80553-f803-4a0e-9b67-3df8fed0ad4d (at 192.168.204.35@tcp) reconnecting [ 4219.251228] Lustre: *** cfs_fail_loc=159, val=0*** [ 4235.844041] Lustre: lustre-MDT0001: Client 5af80553-f803-4a0e-9b67-3df8fed0ad4d (at 192.168.204.35@tcp) reconnecting [ 4237.220543] Lustre: *** cfs_fail_loc=159, val=0*** [ 4237.223407] Lustre: Skipped 3 previous similar messages [ 4253.722298] Lustre: lustre-MDT0001: Client 5af80553-f803-4a0e-9b67-3df8fed0ad4d (at 192.168.204.35@tcp) reconnecting [ 4258.713975] Lustre: 139208:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4261.507938] Lustre: *** cfs_fail_loc=159, val=0*** [ 4261.514852] Lustre: Skipped 4 previous similar messages [ 4277.339049] Lustre: lustre-MDT0000: Client 39cc504e-5e01-4a3c-8b03-1602eb55e279 (at 192.168.204.35@tcp) reconnecting [ 4279.593961] Lustre: *** cfs_fail_loc=159, val=0*** [ 4279.596295] Lustre: Skipped 14 previous similar messages [ 4295.191598] Lustre: lustre-MDT0000: Client 39cc504e-5e01-4a3c-8b03-1602eb55e279 (at 192.168.204.35@tcp) reconnecting [ 4296.848283] Lustre: 139501:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4315.620708] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4315.631390] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4315.669588] Lustre: Skipped 1 previous similar message [ 4315.674542] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4315.686695] Lustre: Skipped 1 previous similar message [ 4320.740608] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4320.750771] Lustre: Skipped 1 previous similar message [ 4321.785454] Lustre: server umount lustre-OST0000 complete [ 4325.859173] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4325.866428] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4325.873984] Lustre: Skipped 1 previous similar message [ 4330.980692] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4330.991695] Lustre: Skipped 2 previous similar messages [ 4332.006684] Lustre: server umount lustre-MDT0000 complete [ 4335.450237] LustreError: 139328:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786404362 with bad export cookie 17824386089147425658 [ 4335.460885] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4335.472568] LustreError: 139328:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4335.760822] Lustre: server umount lustre-MDT0001 complete [ 4346.199833] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [ 4349.230737] Key type lgssc unregistered [ 4349.510861] LNet: 140581:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4349.526186] LNetError: 140581:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4349.556386] LNet: Removed LNI 192.168.204.135@tcp [ 4350.438253] Key type .llcrypt unregistered [ 4350.440487] Key type ._llcrypt unregistered [ 4376.179227] Key type ._llcrypt registered [ 4376.181038] Key type .llcrypt registered [ 4376.312853] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 4377.598848] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4377.654098] alg: No test for adler32 (adler32-zlib) [ 4378.980596] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [ 4379.424179] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [ 4381.137244] Key type lgssc registered [ 4382.642557] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4391.285860] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4391.300430] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4392.605314] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4398.374301] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 4402.162569] LustreError: 141995:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4402.190933] LustreError: 141995:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4407.265074] LustreError: 141996:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4407.653463] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4408.190348] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4413.648451] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 4422.821909] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4429.535936] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4437.001557] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4437.280388] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4438.352946] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4445.129346] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 4452.715316] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4453.772719] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4459.530338] Lustre: 144576:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4462.556174] Lustre: 144625:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4463.948454] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4464.741850] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4468.378429] Lustre: 144771:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4469.314106] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4470.235845] Lustre: 144879:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4474.338728] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4474.351226] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4474.368987] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4478.434385] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4478.435389] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4478.458052] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4480.423714] Lustre: server umount lustre-OST0000 complete [ 4485.088168] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4485.101996] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4485.111784] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4485.120946] Lustre: Skipped 1 previous similar message [ 4488.674508] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4488.677054] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4490.465706] Lustre: server umount lustre-MDT0000 complete [ 4493.802446] LustreError: 141995:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4493.842604] LustreError: 141995:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4494.529450] LustreError: 141976:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786404521 with bad export cookie 12942686645392657611 [ 4494.560599] LustreError: 141976:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4494.561988] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4494.823534] Lustre: server umount lustre-MDT0001 complete [ 4505.805122] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [ 4508.423979] Key type lgssc unregistered [ 4508.785384] LNet: 145929:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4508.801798] LNetError: 145929:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4509.861920] LNet: Removed LNI 192.168.204.135@tcp [ 4510.884850] Key type .llcrypt unregistered [ 4510.888962] Key type ._llcrypt unregistered [ 4537.422848] Key type ._llcrypt registered [ 4537.430390] Key type .llcrypt registered [ 4537.594291] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 4538.839270] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4538.891297] alg: No test for adler32 (adler32-zlib) [ 4539.966537] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [ 4540.368993] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [ 4542.127726] Key type lgssc registered [ 4543.834447] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4553.232408] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4553.265313] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4554.650503] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4559.892639] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 4564.968669] LustreError: 147348:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4564.994561] LustreError: 147348:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4569.718606] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4570.185820] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4576.449067] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 4584.253583] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4591.724413] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4601.009791] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4601.410480] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4609.019898] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 4615.734347] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4617.552416] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4624.950163] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4633.204917] Lustre: *** cfs_fail_loc=159, val=0*** [ 4649.502567] Lustre: lustre-MDT0000: Client a68c0ebd-ff69-4a62-8a42-2f3c8f21a5c1 (at 192.168.204.35@tcp) reconnecting [ 4666.850062] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4666.858633] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4666.877182] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4668.495558] Lustre: server umount lustre-OST0000 complete [ 4672.481526] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4672.488969] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4672.500070] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4676.067497] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4676.069939] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4676.100509] Lustre: Skipped 1 previous similar message [ 4681.186617] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4681.201456] Lustre: Skipped 1 previous similar message [ 4682.306920] Lustre: server umount lustre-MDT0000 complete [ 4685.288810] LustreError: 148272:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4685.328699] LustreError: 148272:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4686.733738] LustreError: 147327:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786404713 with bad export cookie 7989671716052703709 [ 4686.766547] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4686.767245] LustreError: 147327:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4687.203673] Lustre: server umount lustre-MDT0001 complete [ 4699.553666] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [ 4703.155814] Key type lgssc unregistered [ 4703.618710] LNet: 151035:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4703.630671] LNetError: 151035:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4703.652637] LNet: Removed LNI 192.168.204.135@tcp [ 4704.570417] Key type .llcrypt unregistered [ 4704.572643] Key type ._llcrypt unregistered [ 4733.461758] Key type ._llcrypt registered [ 4733.467698] Key type .llcrypt registered [ 4733.651974] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 4735.313776] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4735.343557] alg: No test for adler32 (adler32-zlib) [ 4736.462949] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [ 4736.796178] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [ 4738.471194] Key type lgssc registered [ 4740.247981] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4750.359469] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4750.375088] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4751.887367] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4757.550352] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 4762.124307] LustreError: 152451:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4762.164185] LustreError: 152451:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4767.216375] LustreError: 152452:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4768.739480] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4769.219671] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4774.975706] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 4785.176678] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4791.865351] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4799.154039] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4799.601112] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4807.185811] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 4814.956087] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4815.270645] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4822.205051] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4844.280643] Lustre: lustre-MDT0001: evicting 075d4c10-2534-423d-a713-bcd3bacd7efa (at 192.168.204.35@tcp) by administrative request [ 4844.286373] Lustre: Skipped 1 previous similar message [ 4873.702697] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4873.709628] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4873.712606] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4873.723763] Lustre: Skipped 1 previous similar message [ 4878.829337] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4878.864202] Lustre: Skipped 2 previous similar messages [ 4879.311687] Lustre: server umount lustre-OST0000 complete [ 4883.957202] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4883.977671] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4883.990729] Lustre: Skipped 1 previous similar message [ 4889.061431] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4889.073059] Lustre: Skipped 2 previous similar messages [ 4890.481064] Lustre: server umount lustre-MDT0000 complete [ 4894.197996] LustreError: 152451:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4894.226246] LustreError: 152451:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4894.667204] LustreError: 152432:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786404921 with bad export cookie 2490708029557416332 [ 4894.675969] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4894.676361] LustreError: 152432:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4895.083401] Lustre: server umount lustre-MDT0001 complete [ 4908.003717] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [ 4911.611763] Key type lgssc unregistered [ 4911.980782] LNet: 156550:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4911.992336] LNetError: 156550:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4912.006886] LNet: Removed LNI 192.168.204.135@tcp [ 4913.200090] Key type .llcrypt unregistered [ 4913.205677] Key type ._llcrypt unregistered [ 4998.275316] Key type ._llcrypt registered [ 4998.278832] Key type .llcrypt registered [ 4998.508597] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 5000.691795] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5000.746284] alg: No test for adler32 (adler32-zlib) [ 5002.259850] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [ 5002.655194] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [ 5004.335196] Key type lgssc registered [ 5006.282624] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5016.351104] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5016.375735] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5017.919218] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5023.668764] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 5027.311572] LustreError: 159042:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5027.348736] LustreError: 159042:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5032.418020] LustreError: 159043:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5033.385942] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5033.683657] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5038.524273] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 5046.570179] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5052.177321] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5059.334828] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5059.849890] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5062.931405] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5067.185895] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 5074.310449] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5078.323754] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5087.719426] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 5087.731101] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5087.752572] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5088.735994] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5088.743976] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5088.747820] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5088.764674] Lustre: Skipped 1 previous similar message [ 5092.028226] Lustre: server umount lustre-OST0000 complete [ 5098.984705] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5099.007995] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5099.018385] Lustre: Skipped 1 previous similar message [ 5102.474376] Lustre: server umount lustre-MDT0000 complete [ 5104.098617] LustreError: 159039:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5104.109186] LustreError: 159039:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5106.828424] LustreError: 159025:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786405133 with bad export cookie 3155118803025276844 [ 5106.837124] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5106.844920] LustreError: 159025:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5107.202624] Lustre: server umount lustre-MDT0001 complete [ 5121.566704] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [ 5125.083445] Key type lgssc unregistered [ 5125.577204] LNet: 162582:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5125.587588] LNetError: 162582:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5126.631565] LNet: Removed LNI 192.168.204.135@tcp [ 5127.878459] Key type .llcrypt unregistered [ 5127.880326] Key type ._llcrypt unregistered [ 5157.439211] Key type ._llcrypt registered [ 5157.450488] Key type .llcrypt registered [ 5157.719665] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 5159.592799] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5159.638037] alg: No test for adler32 (adler32-zlib) [ 5160.855297] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [ 5161.233244] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [ 5162.967171] Key type lgssc registered [ 5164.443830] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5182.304205] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 19:40:09 (1786405209) [ 5196.615285] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_hostid [ 5208.195406] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 5247.473085] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5247.552260] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5248.852904] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5248.905915] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5249.014758] Lustre: lustre-MDT0000: new disk, initializing [ 5249.118239] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5249.157163] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5254.366519] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 5266.557537] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5266.689738] Lustre: 166859:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5266.755692] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5266.760046] Lustre: Skipped 1 previous similar message [ 5266.855895] Lustre: lustre-MDT0001: new disk, initializing [ 5266.937515] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5266.992371] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5267.027300] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5272.822396] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 5283.385871] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5288.393354] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5294.966937] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5295.220388] Lustre: lustre-OST0000: new disk, initializing [ 5295.226929] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5295.242540] Lustre: 168826:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5295.352329] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5301.418845] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5301.449436] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5301.634813] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5301.829321] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 5310.897780] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5322.211388] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5322.217101] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5322.230226] LustreError: Skipped 1 previous similar message [ 5322.238322] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5322.247331] Lustre: Skipped 1 previous similar message [ 5322.267312] Lustre: Skipped 1 previous similar message [ 5326.183641] Lustre: server umount lustre-OST0000 complete [ 5332.451622] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5332.462425] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5332.466572] Lustre: Skipped 1 previous similar message [ 5336.934272] Lustre: server umount lustre-MDT0000 complete [ 5341.222959] LustreError: 166850:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786405368 with bad export cookie 11422595090752077078 [ 5341.244179] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5341.254757] LustreError: 166850:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5341.697978] Lustre: server umount lustre-MDT0001 complete [ 5354.761286] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [ 5359.240887] Key type lgssc unregistered [ 5359.665458] LNet: 170733:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5359.691201] LNetError: 170733:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5360.742367] LNet: Removed LNI 192.168.204.135@tcp [ 5361.958401] Key type .llcrypt unregistered [ 5361.964243] Key type ._llcrypt unregistered [ 5393.926444] Key type ._llcrypt registered [ 5393.935694] Key type .llcrypt registered [ 5394.081930] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 5395.319339] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5395.363237] alg: No test for adler32 (adler32-zlib) [ 5396.384123] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [ 5396.586581] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [ 5398.247657] Key type lgssc registered [ 5399.512594] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5407.787755] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5407.805339] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5409.182169] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5414.506265] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 5419.493186] LustreError: 172149:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5419.529828] LustreError: 172149:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5424.612405] LustreError: 172150:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5425.999632] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5426.408328] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5432.783550] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 5442.123988] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5448.937912] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5456.192956] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5456.541736] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5463.497990] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 5472.425243] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5479.093882] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5486.124165] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5486.144803] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5486.170987] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5486.190701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.252364] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5487.838561] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5487.876522] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5487.903376] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5487.929873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.957162] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5489.622938] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5489.695581] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5489.718296] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5489.723781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.734766] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5490.888739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.851349] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5491.858189] Lustre: Skipped 1 previous similar message [ 5491.870579] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5491.880825] Lustre: Skipped 1 previous similar message [ 5491.884554] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5491.888935] Lustre: Skipped 1 previous similar message [ 5491.891581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.898514] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5491.904955] Lustre: Skipped 1 previous similar message [ 5492.865413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.129741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.229575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.215852] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5496.219505] Lustre: Skipped 3 previous similar messages [ 5496.239821] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5496.257681] Lustre: Skipped 3 previous similar messages [ 5496.261872] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5496.268595] Lustre: Skipped 3 previous similar messages [ 5496.275333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.284603] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5496.290541] Lustre: Skipped 3 previous similar messages [ 5497.220221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.617614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.044532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.118475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.137046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.977539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.073834] LustreError: 172169:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5504.107142] LustreError: 172169:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5504.131419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.410706] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5505.420688] Lustre: Skipped 7 previous similar messages [ 5505.446335] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5505.460356] Lustre: Skipped 7 previous similar messages [ 5505.464930] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5505.473955] Lustre: Skipped 7 previous similar messages [ 5505.477955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.493689] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5505.508251] Lustre: Skipped 7 previous similar messages [ 5506.713632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.795269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.698455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.959563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.730471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.825309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.730818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.835995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.847964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.951663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.862917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.908106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.933526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.276705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.027233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.947403] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5521.953751] Lustre: Skipped 15 previous similar messages [ 5521.972447] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5521.980156] Lustre: Skipped 15 previous similar messages [ 5521.983776] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5521.987742] Lustre: Skipped 15 previous similar messages [ 5521.990881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.998074] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5522.005253] Lustre: Skipped 15 previous similar messages [ 5522.757980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.657771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.562821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.365519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.328848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.637957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.482613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.622097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.511567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.725926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.874577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.065177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.191861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.059230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.402329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.372713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.686460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.601150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.864910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.856544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.971574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.856615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.921678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.912253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.134328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.372426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.160688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.165201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.094558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.990732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.030558] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5554.043313] Lustre: Skipped 30 previous similar messages [ 5554.082427] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5554.093088] Lustre: Skipped 30 previous similar messages [ 5554.099132] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5554.104948] Lustre: Skipped 30 previous similar messages [ 5554.108761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.121116] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5554.130166] Lustre: Skipped 30 previous similar messages [ 5555.489428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.990384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.035970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.000032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.516610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.683485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.719340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.104370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.541057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.492266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.607836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.500231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.526413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.607361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.491231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.699088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.631989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.748440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.686942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.056124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.904812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.803545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.641064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.783210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.629950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.810143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.871808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.768475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.790829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.711809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.965368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.189494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.181986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.197514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.142764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.615916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.738293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.767702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.873804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.932853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.729627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.534286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.151911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.732133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.763568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.135560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.934578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.083081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.141628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.394055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.055396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.112347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.108827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.443558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.646431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.574536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.897444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.304458] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5618.314755] Lustre: Skipped 57 previous similar messages [ 5618.347203] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5618.360262] Lustre: Skipped 57 previous similar messages [ 5618.365111] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5618.372151] Lustre: Skipped 57 previous similar messages [ 5618.379683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.393292] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5618.404463] Lustre: Skipped 57 previous similar messages [ 5619.276794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.812840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.201259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.272128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.239391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.444150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.424795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.551577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.527049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.662585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.702864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.224025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.146069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.827436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.949719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.178705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.138501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.610059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.617978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.001681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.130425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.365843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.342711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.670902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.266808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.452832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.599821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.059075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.966283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.168487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.469155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.689989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.791172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.146635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.691265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.008441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.878792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.239321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.385135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.762152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.973920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.075759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.502358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.730125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.934858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.682182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.771040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.067985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.628968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.206804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.603761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.981703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.481458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.399855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.404047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.212605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.571840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.620718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.098273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.186276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.249558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.482575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.797558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.948956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.282656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.375957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.541624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.658139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.812616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.106942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.358071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.784900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.896734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.852108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.004363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.900826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.048137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.113883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.392934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.765151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.147515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.294417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.625871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.894345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.953809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.836554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.956870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.900200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.492854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.691073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.916434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.139466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.554651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.078522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.273251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.663465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.988829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.203597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.383714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.464532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.659369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.582351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.696287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.598941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.559140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.657606] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5746.672734] Lustre: Skipped 105 previous similar messages [ 5746.702732] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5746.711084] Lustre: Skipped 105 previous similar messages [ 5746.713878] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5746.719658] Lustre: Skipped 105 previous similar messages [ 5746.722812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.730955] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5746.736173] Lustre: Skipped 105 previous similar messages [ 5748.174601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.253969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.377579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.336176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.472988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.597766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.559562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.680834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.702086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.697295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.930889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.290594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.401836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.432960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.275531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.354651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.272504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.322323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.144986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.006496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.789373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.891460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.750866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.963171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.106657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.081529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.237755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.486225] LustreError: 172169:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5776.511790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.515949] LustreError: 172169:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5777.761549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.758705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.125927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.215320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.357961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.358480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.555876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.336637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.380735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.221517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.132249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.074996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.453969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.498198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.492383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.718527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.721199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.715064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.657260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.671210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.538740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.743603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.589483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.699191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.717412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.029809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.885432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.069471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.128568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.081895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.220622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.130801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.234403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.186768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.148397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.931968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.577218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.662134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.465157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.511883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.849191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.898664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.138983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.123360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.516125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.814626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.907392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.799689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.115396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.072501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.478805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.480571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.663721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.060925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.260657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.379732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.564282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.893592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.114549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.761844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.024632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.090903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.284719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.516979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.690235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.463121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.962491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.034120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.986741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.214062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.251676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.124686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.446918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.440394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.526811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.331673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.797124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.148671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.853277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.124582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.049208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.333664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.512173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.814054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.949641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.019901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.311045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.462974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.874692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.375666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.444331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.387485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.392970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.740882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.934372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.029955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.014912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.370161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.385477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.975517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.885725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.143345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.613337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.623323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.702933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.822466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.021752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.250554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.130163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.314323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.328873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.376066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.263412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.454836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.498676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.299975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.280071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.594038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.402278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.481463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.365344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.688276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.238313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.347680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.601714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.945401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.839594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.202602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.169636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.351171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.273196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.424132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.694363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.913289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.989993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.132773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.168545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.456221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.643469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.065514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.036806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.355996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.376913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.541088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.584164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.001660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.025465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.173244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.061666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.226869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.447349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.439573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.396053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.593514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.567902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.966155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.045480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.140587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.410421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.671903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.109953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.020369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.434076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.839438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.250141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.397863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.810779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.046208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.258748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.464154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.527227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.710970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.692436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.163573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.267169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.538874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.588981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.834622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.841972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.231231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.280774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.392763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.378631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.331916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.345330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.168980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.246109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.248995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.272068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.431548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.415922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.691594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.482169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.648430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.785716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.987502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.078537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.047983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.252278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.259288] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6003.268312] Lustre: Skipped 227 previous similar messages [ 6003.287555] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6003.304894] Lustre: Skipped 227 previous similar messages [ 6003.311407] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6003.322156] Lustre: Skipped 227 previous similar messages [ 6003.328765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.349150] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6003.362680] Lustre: Skipped 227 previous similar messages [ 6004.370474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.341955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.550580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.483512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.488719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.437452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.286599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.314927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.246477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.201848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.672438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.007800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.889951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.043245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.208428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.212629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.264867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.242390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.552625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.688729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.056333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.267795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.222643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.741725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.088247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.291475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.496918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.295133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.687542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.891127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.165401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.481340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.003866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.085449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.329090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.549319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.531340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.895389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.033120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.983682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.408086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.511061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.439295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.630342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.566234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.830759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.209331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.249048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.339992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.277213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.458829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.577454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.442472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.585892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.483485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.847644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.698535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.853494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.097638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.665131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.916738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.197215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.375382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.931623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.970801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.478549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.121222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.058927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.422786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.917249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.288938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.613253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.588485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.905661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.184055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.533579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.372708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.487264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.708392] LustreError: 172169:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6095.723201] LustreError: 172169:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6095.726959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.755197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.775925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.609628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.648272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.404245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.521336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.611053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.087429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.504415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.685229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.920168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.009973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.916396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.346284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.794768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.452268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.568224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.497967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.979871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.282663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.615246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.432789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.402300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.242683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.244598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.031778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.171925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.787969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.368856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.613871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.089102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.613132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.610657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.130269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.348899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.611894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.497850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.038703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.388973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.740680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.770029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.905481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.831790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.068424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.049840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.234229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.451185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.685466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.820934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.170233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.382267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.734368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.867360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.404538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.771611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.608678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.296324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.442361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.458145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.779076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.682051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.746994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.815768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.197464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.391868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.674226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.993600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.938977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.394658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.881316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.394209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.377360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.920088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.040666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.314896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.453439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.479538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.961996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.009051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.155124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.394860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.536381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.694225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.151484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.710765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.549032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.040645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.528868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.215274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.733712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.133099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.558091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.280022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.478738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.596308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.759129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.115417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.297157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.442621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.801950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.662627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.867325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.739039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.979393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.879632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.672901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.607296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.132704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.261381] LustreError: 172169:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6229.275080] LustreError: 172169:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6229.311353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.244743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.348640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.256765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.556834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.025882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.251332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.843162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.519193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.863863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.881848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.225888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.441736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.389462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.538762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.593044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.699358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.313417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.551975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.786129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.367569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.015065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.455707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.720950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.433450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.677662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.946701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.653507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.819659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.980698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.065417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.644354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.339408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.539198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.197362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.431103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.485800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.501336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.802722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.708156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.950266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.042422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.267979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.231731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.482973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.678755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.086192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.463549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.744584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.135226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.331794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.899572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.426267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.441623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.761075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.362694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.400453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.027543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.737806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.911546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.243793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.194709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.442597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.737320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.095492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.306952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.509343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.811360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.878502] LustreError: 172169:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6316.897298] LustreError: 172169:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6316.908868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.908760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.227714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.360783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.377339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.639902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.983527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.163939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.196678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.244066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.309428] LustreError: 172169:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6328.319153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.324386] LustreError: 172169:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6329.433027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.594775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.604421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.817868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.849721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.318755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.586755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.736180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.279100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.563727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.636844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.383344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.535652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.310574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.156975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.300408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.212747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.097061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.459756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.664457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.582346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.795249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.963183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.284468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.317742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.556589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.044300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.425863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.615635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.749559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.100476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.330894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.378610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.343409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.369688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.673202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.027643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.599722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.069422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.408070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.687036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.121319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.097474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.316341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.348163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.596925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.652262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.778313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.063619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.366895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.524533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.699438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.613484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.681721] LustreError: 172169:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6391.695791] LustreError: 172169:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6391.709309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.684472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.807685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.114355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.037957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.232502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.502557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.489115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.691364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.251883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.256779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.478367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.997583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.034436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.000589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.258583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.435178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.313946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.768675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.941615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.264532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.272090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.486936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.543593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.478964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.839498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.840989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.169514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.047616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.178434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.386873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.512270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.504659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.124096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.650606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.489853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.405994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.730734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.332521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.361222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.872073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.709184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.948633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.985741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.624948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.093901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.346298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.380480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.525048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.589533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.995860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.316439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.175758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.231623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.274359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.517042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.867857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.012486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.478679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.949550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.996988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.197903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.432296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.421486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.038447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.020417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.201469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.285932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.389110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.451878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.664826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.904149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.353805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.593037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.510822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.576284] LustreError: 172169:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6482.584470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.587736] LustreError: 172169:0:(osp_precreate.c:655:osp_precreate_send()) Skipped 1 previous similar message [ 6482.587779] LustreError: 172169:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6482.587784] LustreError: 172169:0:(osp_precreate.c:1359:osp_precreate_thread()) Skipped 1 previous similar message [ 6483.718891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.009335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.985157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.342985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.363890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.337565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.607926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.386160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.949214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.238450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.764657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.784131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.090820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.580105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.412905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.788782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.006757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.021648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.364419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.504045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.787473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.037387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.126736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.085081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.522816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.312544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.521192] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6515.541994] Lustre: Skipped 421 previous similar messages [ 6515.576400] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6515.600554] Lustre: Skipped 421 previous similar messages [ 6515.605360] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6515.615194] Lustre: Skipped 421 previous similar messages [ 6515.621869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.636294] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6515.647335] Lustre: Skipped 421 previous similar messages [ 6516.937363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.153029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.159191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6520.627424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.104026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.612918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.892288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6526.007761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.357167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.328352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6529.918744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6530.991474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.624298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.764442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6534.868909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.447960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.477432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.108473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.658727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.244253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.735600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.700992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.170927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.654488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.818468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.092805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.154638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6552.292946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.546521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.995146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.057224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.285617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.295276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.649348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.642504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.952520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.913089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.021494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.567570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.584798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.774552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.700743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.656950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.926558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.283243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.263067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.507411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.632787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.708156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.861774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.095434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.675278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.160895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.486754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.018599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6586.353470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.783166] LustreError: 172169:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6587.810138] LustreError: 172169:0:(osp_precreate.c:655:osp_precreate_send()) Skipped 1 previous similar message [ 6587.823566] LustreError: 172169:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6587.835090] LustreError: 172169:0:(osp_precreate.c:1359:osp_precreate_thread()) Skipped 1 previous similar message [ 6587.842570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.621841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.016573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6592.185560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6593.118679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.413543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.512469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.861444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.322150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.701269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.982470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.178720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.585476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.673622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.566424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6607.037108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.243361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.569410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.410530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.581675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.596710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.447365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.421916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6615.600298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6616.866449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.006500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.989762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.252221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6621.354985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.431983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6623.365548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.646449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6625.444610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6626.230987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.202339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.254932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.486469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6630.434257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6631.447388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6632.377112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6633.578706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6634.412782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6636.135076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6637.276223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6638.389136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.952790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6641.006308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6642.447503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6643.881759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6645.099498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6646.361916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.656793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6649.096153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.484401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6651.203749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6652.699912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.354753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.752280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6656.740577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6658.305538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.210401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6660.836469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.867116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6663.095208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6664.505278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6665.786230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7065.209404] LustreError: 173971:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7065.219763] LustreError: 172169:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7753.704343] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7753.719125] Lustre: Skipped 122 previous similar messages [ 7753.726633] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7753.744104] Lustre: Skipped 1 previous similar message [ 7758.170560] Lustre: server umount lustre-OST0000 complete [ 7763.945819] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7763.957769] Lustre: Skipped 1 previous similar message [ 7768.803677] Lustre: server umount lustre-MDT0000 complete [ 7769.066071] LustreError: 173853:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7769.075587] LustreError: 173853:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7772.405809] LustreError: 172130:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786407799 with bad export cookie 12156098310962876747 [ 7772.408509] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7772.424050] LustreError: 172130:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7772.927070] Lustre: server umount lustre-MDT0001 complete [ 7784.063275] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [ 7786.772880] Key type lgssc unregistered [ 7787.057045] LNet: 227479:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7787.071207] LNetError: 227479:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7787.087711] LNet: Removed LNI 192.168.204.135@tcp [ 7788.045282] Key type .llcrypt unregistered [ 7788.047264] Key type ._llcrypt unregistered [ 7817.715621] Key type ._llcrypt registered [ 7817.719781] Key type .llcrypt registered [ 7817.852754] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 7819.081569] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7819.133919] alg: No test for adler32 (adler32-zlib) [ 7820.339641] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [ 7820.574790] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [ 7822.263218] Key type lgssc registered [ 7823.496185] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7831.058941] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7831.087166] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7832.484885] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7836.764663] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 7842.788653] LustreError: 228896:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7842.803184] LustreError: 228896:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7846.271737] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7846.542480] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7850.927499] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 7859.405395] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7866.036857] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7873.183869] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7873.583322] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7877.707472] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7881.652331] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 7890.093626] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7896.050864] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7898.094112] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7898.113921] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7898.127872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7898.140235] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7898.140589] LustreError: 228916:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7902.180395] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7902.190939] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7902.217472] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7903.248758] Lustre: lustre-OST0000: Not available for connect from 192.168.204.35@tcp (stopping) [ 7905.252613] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7905.267624] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7906.697254] Lustre: server umount lustre-OST0000 complete [ 7918.199182] LustreError: 228916:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7941.718786] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7941.960228] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7946.101408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7946.117562] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7948.077718] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 7955.118853] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7955.368229] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7955.390931] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7957.109903] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7963.074721] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff94097de22800.ost_server_uuid 50 [ 7964.621772] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff94097de22800.ost_server_uuid in FULL state after 0 sec [ 7970.273813] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7970.283611] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7970.300793] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7970.310818] Lustre: Skipped 1 previous similar message [ 7975.549798] Lustre: server umount lustre-OST0000 complete [ 7981.026453] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7981.035196] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7981.039358] Lustre: Skipped 2 previous similar messages [ 7981.048405] Lustre: Skipped 1 previous similar message [ 7986.013307] Lustre: server umount lustre-MDT0000 complete [ 7986.149072] LustreError: 230903:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7986.201264] LustreError: 230903:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7990.244300] LustreError: 230903:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7990.265026] LustreError: 230903:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7990.983073] LustreError: 228876:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786408017 with bad export cookie 5565908770350208760 [ 7990.985396] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7991.009109] LustreError: 228876:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7991.417766] Lustre: server umount lustre-MDT0001 complete [ 8006.414920] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [ 8011.302498] Key type lgssc unregistered [ 8011.798276] LNet: 234105:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8011.805084] LNetError: 234105:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8012.841651] LNet: Removed LNI 192.168.204.135@tcp [ 8014.076528] Key type .llcrypt unregistered [ 8014.084496] Key type ._llcrypt unregistered [ 8061.941759] Key type ._llcrypt registered [ 8061.944991] Key type .llcrypt registered [ 8062.121315] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 8064.083907] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8064.281415] alg: No test for adler32 (adler32-zlib) [ 8065.775975] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [ 8066.238102] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [ 8068.127183] Key type lgssc registered [ 8069.921192] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8083.621376] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8083.650987] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8085.036831] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8090.675686] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 8094.188571] LustreError: 235840:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8094.241576] LustreError: 235840:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8099.296893] LustreError: 235841:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8101.588338] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8102.021289] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8107.220449] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 8110.336509] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8118.006142] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8118.340404] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8125.755280] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 8129.690065] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 8137.464934] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8137.764878] Lustre: lustre-OST0001: new disk, initializing [ 8137.773669] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8137.780026] Lustre: 238150:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8137.930893] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8146.571966] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 8148.153349] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8148.164808] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8148.371256] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8159.284310] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8166.720990] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8183.781274] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8183.793831] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8183.813558] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8184.306660] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8184.316083] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8184.327686] Lustre: Skipped 2 previous similar messages [ 8184.337025] Lustre: Skipped 2 previous similar messages [ 8187.724891] Lustre: server umount lustre-MDT0000 complete [ 8189.418069] LustreError: 235841:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8189.446833] LustreError: 235841:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8192.355050] LustreError: 235821:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786408219 with bad export cookie 16874675201589746773 [ 8192.360370] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8192.387379] LustreError: 235821:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8192.928427] Lustre: server umount lustre-MDT0001 complete [ 8207.649879] Lustre: server umount lustre-OST0000 complete [ 8210.911534] Lustre: 235102:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786408221/real 1786408221] req@ffff988e46d7ce00 x1873184652625536/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786408237 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8210.944849] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8213.162367] Lustre: server umount lustre-OST0001 complete [ 8243.035245] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 8258.137233] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8258.798966] LustreError: 241260:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8258.932238] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8264.177369] LustreError: 241261:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8265.386173] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 8273.387523] LustreError: 241261:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8273.409649] LustreError: 241261:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8275.294069] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8275.909287] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8281.354276] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 8284.808790] Lustre: 242402:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8291.111828] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8297.484507] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 8300.741650] LustreError: 242758:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8300.812508] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 8303.815945] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 8308.507057] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8308.820990] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8308.828314] Lustre: Skipped 1 previous similar message [ 8313.889558] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 8317.288206] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 8325.987525] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8330.901287] Lustre: 244271:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8344.547803] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8344.568652] Lustre: Skipped 1 previous similar message [ 8344.574222] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8344.594194] Lustre: Skipped 2 previous similar messages [ 8349.674098] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8349.683695] Lustre: Skipped 3 previous similar messages [ 8354.784227] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8354.800719] Lustre: Skipped 3 previous similar messages [ 8358.367163] Lustre: mylustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8358.661578] Lustre: server umount mylustre-MDT0000 complete [ 8359.914210] LustreError: 241255:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8359.942426] LustreError: 241255:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 8363.452378] LustreError: 244274:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786408390 with bad export cookie 16874675201589749636 [ 8363.467460] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8363.708566] Lustre: server umount mylustre-MDT0001 complete [ 8377.874038] Lustre: server umount mylustre-OST0000 complete [ 8380.197356] Lustre: 235101:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786408391/real 1786408391] req@ffff988e46d7e680 x1873184652693504/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786408407 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8380.268636] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8380.312522] Lustre: Skipped 2 previous similar messages [ 8383.024343] Lustre: server umount mylustre-OST0001 complete [ 8409.672792] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 8423.944178] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8424.709605] LustreError: 246586:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8424.907185] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8430.978381] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 8441.542419] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8448.107794] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 8451.955701] Lustre: 247731:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8460.478714] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8467.100969] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 8468.970418] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 8469.179574] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 8480.342726] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8485.881187] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:65) [ 8489.147895] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 8498.340644] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8502.945180] Lustre: 249601:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8511.463563] Lustre: tfs-MDT0000-lwp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8511.480804] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8511.499803] Lustre: Skipped 3 previous similar messages [ 8511.518773] Lustre: Skipped 3 previous similar messages [ 8517.488727] Lustre: server umount tfs-MDT0000 complete [ 8521.483466] LustreError: 246561:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786408548 with bad export cookie 16874675201589751813 [ 8521.492051] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8521.502712] LustreError: 246561:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 8521.921688] Lustre: server umount tfs-MDT0001 complete [ 8528.517586] Lustre: server umount tfs-OST0000 complete [ 8533.298969] Lustre: server umount tfs-OST0001 complete [ 8561.228589] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 8576.369537] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8576.754347] LustreError: 251852:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8576.783108] LustreError: 251852:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 12 previous similar messages [ 8576.872396] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8576.881726] Lustre: Skipped 3 previous similar messages [ 8582.957676] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 8594.460970] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8599.359895] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 8603.190991] Lustre: 252994:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8612.363176] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8620.097435] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 8621.174932] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 8622.157304] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 8633.200253] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8638.982950] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:97) [ 8642.752314] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 8652.956614] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8657.212763] Lustre: 254868:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8665.493434] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 20:38:12 (1786408692) [ 8685.031831] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8685.037529] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8685.045619] Lustre: Skipped 4 previous similar messages [ 8685.058194] Lustre: Skipped 7 previous similar messages [ 8688.585718] Lustre: server umount lustre-MDT0000 complete [ 8692.806633] LustreError: 251827:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786408719 with bad export cookie 16874675201589753724 [ 8692.824374] LustreError: 251827:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8692.826657] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8699.490368] Lustre: server umount lustre-MDT0001 complete [ 8718.304361] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8718.452184] Lustre: server umount lustre-OST0000 complete [ 8729.526547] Lustre: server umount lustre-OST0001 complete [ 8738.642284] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_hostid [ 8752.813293] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 8802.159529] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 8802.159529] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8802.159529] [ 8802.240621] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8802.681473] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8802.781251] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8802.955483] Lustre: lustre-MDT0000: new disk, initializing [ 8803.128267] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8803.138285] Lustre: Skipped 3 previous similar messages [ 8803.206786] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8811.844893] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 8825.924722] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 8825.924722] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8825.924722] [ 8825.960674] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8826.214699] Lustre: 258057:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8826.362231] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8826.372633] Lustre: Skipped 1 previous similar message [ 8826.455729] Lustre: lustre-MDT0001: new disk, initializing [ 8826.819494] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8826.868660] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8833.683855] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 8846.398136] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8846.785807] Lustre: lustre-OST0000: new disk, initializing [ 8846.789132] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8846.794344] Lustre: 259640:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8848.142315] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8848.156971] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8848.293915] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8855.425891] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 8869.354230] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8869.743902] Lustre: lustre-OST0001: new disk, initializing [ 8869.749754] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8869.754794] Lustre: 260513:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8871.363822] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8871.397673] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8871.653543] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8879.338700] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 8894.421803] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 20:42:00 (1786408920) [ 8911.620585] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 20:42:17 (1786408937) [ 8930.277981] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8930.281238] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8930.302339] LustreError: Skipped 1 previous similar message [ 8930.310442] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8930.327314] Lustre: Skipped 4 previous similar messages [ 8930.339915] Lustre: Skipped 7 previous similar messages [ 8932.776697] Lustre: server umount lustre-OST0000 complete [ 8935.911087] LustreError: 259624:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8935.954154] LustreError: 259624:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 19 previous similar messages [ 8939.488199] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8943.874663] Lustre: server umount lustre-MDT0000 complete [ 8949.132506] LustreError: 258048:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786408976 with bad export cookie 16874675201589755509 [ 8949.142872] LustreError: 258048:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8949.149853] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8955.774987] Lustre: server umount lustre-MDT0001 complete [ 8971.153253] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [ 8984.523878] Lustre: server umount lustre-OST0001 complete [ 8993.624534] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_hostid [ 9012.401325] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 9056.972341] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9057.321996] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9057.389774] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9057.455311] Lustre: lustre-MDT0000: new disk, initializing [ 9057.578813] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9062.945986] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 9074.960469] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9075.057845] Lustre: 265075:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9075.129968] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9075.136264] Lustre: Skipped 1 previous similar message [ 9075.193417] Lustre: lustre-MDT0001: new disk, initializing [ 9075.282473] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9075.286855] Lustre: Skipped 4 previous similar messages [ 9075.328172] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9075.339241] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9080.241610] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 9090.880922] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9098.574731] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9106.344918] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9106.765845] Lustre: lustre-OST0000: new disk, initializing [ 9106.780906] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9106.794840] Lustre: 267047:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9108.668843] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9108.690769] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9109.021681] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9115.851280] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 9126.961135] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9134.567446] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9134.572426] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9134.578276] LustreError: Skipped 1 previous similar message [ 9134.587542] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9134.603415] Lustre: Skipped 5 previous similar messages [ 9134.637496] Lustre: Skipped 7 previous similar messages [ 9139.960954] Lustre: server umount lustre-OST0000 complete [ 9150.918828] Lustre: server umount lustre-MDT0000 complete [ 9155.628083] LustreError: 265067:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786409182 with bad export cookie 16874675201589756979 [ 9155.633515] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9155.643481] LustreError: 265067:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9156.209528] Lustre: server umount lustre-MDT0001 complete [ 9170.425486] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [ 9173.762855] Key type lgssc unregistered [ 9174.184206] LNet: 268806:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9174.196099] LNetError: 268806:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9174.213184] LNet: Removed LNI 192.168.204.135@tcp [ 9175.194198] Key type .llcrypt unregistered [ 9175.199429] Key type ._llcrypt unregistered [ 9206.761420] Key type ._llcrypt registered [ 9206.763757] Key type .llcrypt registered [ 9206.895463] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 9209.138884] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9209.196966] alg: No test for adler32 (adler32-zlib) [ 9210.346950] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [ 9210.886990] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [ 9212.623196] Key type lgssc registered [ 9214.516230] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9224.248652] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9224.263502] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9225.990443] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9234.414298] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 9235.435405] LustreError: 270268:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9235.452478] LustreError: 270268:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9240.556113] LustreError: 270269:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9245.666992] LustreError: 270268:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9246.580569] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9247.178270] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9253.719807] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 9262.188663] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9268.715616] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9276.827854] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9277.473952] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9287.517335] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 9295.948095] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 9297.085689] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9298.521627] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9300.963915] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9300.964266] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9300.975098] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9301.002308] Lustre: Skipped 1 previous similar message [ 9302.938347] Lustre: server umount lustre-OST0000 complete [ 9308.644039] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9308.653847] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9308.667126] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9308.671603] Lustre: Skipped 1 previous similar message [ 9311.205339] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9311.214985] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9311.245044] Lustre: Skipped 1 previous similar message [ 9314.322796] Lustre: server umount lustre-MDT0000 complete [ 9316.325777] LustreError: 270269:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9316.340787] LustreError: 270269:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9318.211146] LustreError: 270248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786409345 with bad export cookie 7078820547651161836 [ 9318.224896] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9318.232937] LustreError: 270248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9318.559694] Lustre: server umount lustre-MDT0001 complete [ 9334.044443] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [ 9337.921916] Key type lgssc unregistered [ 9338.302459] LNet: 273856:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9338.315896] LNetError: 273856:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9338.337967] LNet: Removed LNI 192.168.204.135@tcp [ 9339.799226] Key type .llcrypt unregistered [ 9339.801252] Key type ._llcrypt unregistered [ 9361.714920] Key type ._llcrypt registered [ 9361.717925] Key type .llcrypt registered [ 9361.927687] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 9363.460573] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9363.517340] alg: No test for adler32 (adler32-zlib) [ 9364.974260] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [ 9365.280616] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [ 9367.032083] Key type lgssc registered [ 9368.286069] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9377.936422] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9377.959049] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9393.439485] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9398.559393] LustreError: 275051:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.135@tcp: failed processing log, type 4: rc = -110 [ 9425.183199] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9432.933797] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 9441.767329] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9444.930783] Lustre: server umount lustre-OST0000 complete [ 9464.184522] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [ 9467.922865] Key type lgssc unregistered [ 9468.206243] LNet: 276684:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9468.210457] LNetError: 276684:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9468.228675] LNet: Removed LNI 192.168.204.135@tcp [ 9469.258092] Key type .llcrypt unregistered [ 9469.261302] Key type ._llcrypt unregistered [ 9516.445140] Key type ._llcrypt registered [ 9516.447134] Key type .llcrypt registered [ 9516.595704] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 9518.088599] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9518.279962] alg: No test for adler32 (adler32-zlib) [ 9519.971358] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [ 9520.410697] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [ 9522.127207] Key type lgssc registered [ 9523.608393] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9537.636477] loop0: detected capacity change from 0 to 204800000 [ 9537.878109] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9537.889061] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9538.138979] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9539.585235] loop0: detected capacity change from 0 to 204800000 [ 9539.948709] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9539.972571] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9540.464619] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9541.713884] loop0: detected capacity change from 0 to 204800000 [ 9541.916884] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9541.929552] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9542.086967] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9543.335336] loop0: detected capacity change from 0 to 204800000 [ 9543.496487] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9543.504918] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9543.611228] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9544.940733] loop0: detected capacity change from 0 to 204800000 [ 9544.986331] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9549.721375] loop0: detected capacity change from 0 to 204800000 [ 9549.777132] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9554.232295] loop0: detected capacity change from 0 to 204800000 [ 9554.290297] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9558.765433] loop0: detected capacity change from 0 to 204800000 [ 9558.811657] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9564.040311] loop0: detected capacity change from 0 to 204800000 [ 9564.155679] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9564.213260] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9564.334716] Lustre: 280100:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9564.360863] Lustre: 280100:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9564.375805] Lustre: 280100:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9564.390784] Lustre: 280100:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9564.408856] Lustre: 280100:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9564.423938] Lustre: 280100:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9564.445349] Lustre: 280100:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9564.462797] Lustre: 280100:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9564.472810] Lustre: 280100:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9564.485634] Lustre: 280100:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9564.498898] Lustre: 280100:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9564.510492] Lustre: 280100:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9564.521553] Lustre: 280100:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9564.531644] Lustre: 280100:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9570.748771] Lustre: server umount MGS complete [ 9571.842423] loop0: detected capacity change from 0 to 204800000 [ 9571.928719] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9572.286532] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9572.328909] Lustre: lustre-MDT0000: Aborting client recovery [ 9572.390598] LustreError: 280478:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0000-osd: get update log duration 0, retries 0, failed: rc = -108 [ 9573.322251] loop1: detected capacity change from 0 to 204800000 [ 9573.522930] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9573.858781] Lustre: 280535:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9573.878451] Lustre: 280535:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9573.922436] Lustre: 280535:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9573.938398] Lustre: 280535:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9573.957210] Lustre: 280535:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9573.970543] Lustre: 280535:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9573.992519] Lustre: 280535:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9574.029915] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9574.281350] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9574.343676] Lustre: lustre-MDT0001: Aborting client recovery [ 9575.680364] loop2: detected capacity change from 0 to 204800000 [ 9575.917698] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9576.035132] Lustre: 280614:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9576.054140] Lustre: 280614:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9576.072725] Lustre: 280614:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9576.092137] Lustre: 280614:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9576.100673] Lustre: 280614:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9576.110159] Lustre: 280614:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9576.118614] Lustre: 280614:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9576.223613] Lustre: Found index 0 for lustre-OST0000, updating log [ 9576.256322] Lustre: 280633:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9576.327044] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9576.343746] Lustre: lustre-OST0000: Aborting recovery [ 9577.319946] loop3: detected capacity change from 0 to 204800000 [ 9577.591174] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9577.725395] Lustre: 280698:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9577.749188] Lustre: 280698:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9577.767865] Lustre: 280698:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9577.779793] Lustre: 280698:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9577.792382] Lustre: 280698:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9577.801751] Lustre: 280698:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9577.813501] Lustre: 280698:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9577.841774] Lustre: Found index 1 for lustre-OST0001, updating log [ 9577.876705] Lustre: 280707:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9577.972940] Lustre: lustre-OST0001: Aborting recovery [ 9583.097674] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9583.111128] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9583.134475] LustreError: 280479:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 11, retries 0, failed: rc = -108 [ 9583.181274] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9583.216280] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9584.106847] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9584.108950] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9586.021550] LustreError: 280446:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 3935us: rc = -2 [ 9586.533982] LustreError: 280443:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1797us: rc = -2 [ 9586.559271] LustreError: 280443:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 15 previous similar messages [ 9604.580289] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9604.591589] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9604.598886] Lustre: Skipped 2 previous similar messages [ 9604.623353] Lustre: Skipped 3 previous similar messages [ 9607.751281] Lustre: server umount lustre-MDT0000 complete [ 9609.699745] LustreError: 280443:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9609.745353] LustreError: 280443:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 9609.807812] LustreError: 280720:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786409636 with bad export cookie 12571462419824965569 [ 9609.821590] LustreError: 280720:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9609.829807] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9614.823892] LustreError: 280444:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9614.826128] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9614.828860] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9614.868118] LustreError: 280444:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9614.975137] Lustre: Skipped 2 previous similar messages [ 9616.553605] Lustre: server umount lustre-MDT0001 complete [ 9632.225058] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9632.405315] Lustre: server umount lustre-OST0000 complete [ 9640.421298] Lustre: server umount lustre-OST0001 complete [ 9657.338879] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 20:54:42 (1786409682) [ 9702.260403] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_hostid [ 9715.807831] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [ 9756.984929] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9757.260612] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9757.301621] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9757.380432] Lustre: lustre-MDT0000: new disk, initializing [ 9757.464450] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9757.470248] Lustre: Skipped 1 previous similar message [ 9757.484333] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9763.064772] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 9773.881293] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9774.001655] Lustre: 284781:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9774.068242] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9774.072451] Lustre: Skipped 1 previous similar message [ 9774.144682] Lustre: lustre-MDT0001: new disk, initializing [ 9774.221443] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9774.261428] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9774.274733] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9779.508736] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 9790.484383] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9798.495775] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9805.239128] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9805.782321] Lustre: lustre-OST0000: new disk, initializing [ 9805.799601] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9805.813698] Lustre: 286752:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9806.033747] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9811.578047] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9811.592628] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9811.726824] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9813.546448] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 9823.994348] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9829.880497] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9843.013240] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9851.423665] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9898.977666] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9898.983202] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9898.986813] LustreError: Skipped 1 previous similar message [ 9898.991152] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9899.015105] Lustre: Skipped 1 previous similar message [ 9899.049308] Lustre: Skipped 2 previous similar messages [ 9904.102471] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9904.110321] Lustre: Skipped 1 previous similar message [ 9909.227065] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9909.239874] Lustre: Skipped 1 previous similar message [ 9911.263262] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9911.492449] Lustre: server umount lustre-OST0000 complete [ 9917.924800] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9917.935724] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9917.955172] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9931.231262] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9931.613455] Lustre: server umount lustre-MDT0000 complete [ 9934.821289] LustreError: 284792:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9934.843470] LustreError: 284792:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9935.664277] LustreError: 284774:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786409962 with bad export cookie 12571462419824968376 [ 9935.670663] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9935.690179] LustreError: 284774:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9936.145560] Lustre: server umount lustre-MDT0001 complete [ 9946.778020] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9951.540510] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 9957.033951] Lustre: server umount MGS complete [ 9972.851571] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9973.358626] LustreError: 290721:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9973.386987] LustreError: 290734:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9973.446757] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9978.876576] LustreError: 290722:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9979.903249] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [ 9989.090990] LustreError: 290721:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9989.100090] LustreError: 290721:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9989.319934] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9992.984158] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [10002.432843] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10008.782636] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10017.856567] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10018.285888] LustreError: 292546:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10018.567156] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10018.583257] Lustre: Skipped 1 previous similar message [10027.029899] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [10036.774757] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10069.992439] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10069.996027] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10070.018957] LustreError: Skipped 1 previous similar message [10070.024674] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10070.037567] Lustre: Skipped 2 previous similar messages [10070.079768] Lustre: Skipped 6 previous similar messages [10073.891677] Lustre: server umount lustre-OST0000 complete [10080.229825] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10080.250782] Lustre: Skipped 1 previous similar message [10084.806329] Lustre: server umount lustre-MDT0000 complete [10085.358781] LustreError: 291465:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10085.387896] LustreError: 291465:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10089.636328] LustreError: 290702:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786410116 with bad export cookie 12571462419824970315 [10089.651363] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10089.652236] LustreError: 290702:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10090.133911] Lustre: server umount lustre-MDT0001 complete [10104.203852] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [10108.716454] Key type lgssc unregistered [10109.094465] LNet: 294682:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10109.100381] LNetError: 294682:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10109.118563] LNet: Removed LNI 192.168.204.135@tcp [10110.367669] Key type .llcrypt unregistered [10110.369853] Key type ._llcrypt unregistered [10155.378684] Key type ._llcrypt registered [10155.383642] Key type .llcrypt registered [10155.487909] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_hostid [10174.775311] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [10177.738086] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10177.811174] alg: No test for adler32 (adler32-zlib) [10178.957690] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [10179.194908] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [10180.855252] Key type lgssc registered [10182.694415] Lustre: Echo OBD driver; http://www.lustre.org/ [10225.234954] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10225.254606] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10226.535087] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10226.585879] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10226.729510] Lustre: lustre-MDT0000: new disk, initializing [10226.874891] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10226.935951] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10232.969371] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [10246.912159] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10247.087492] Lustre: 298649:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10247.230192] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10247.243622] Lustre: Skipped 1 previous similar message [10247.383621] Lustre: lustre-MDT0001: new disk, initializing [10247.470165] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10247.514909] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10247.534691] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10253.715631] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [10264.280897] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10270.498128] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10278.644093] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10278.928633] Lustre: lustre-OST0000: new disk, initializing [10278.935783] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10278.946505] Lustre: 300618:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10279.045336] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10285.650704] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10285.665901] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10285.787409] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10286.341968] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [10296.987576] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10304.622322] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10317.537830] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10330.854448] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10335.964209] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10378.218764] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10378.221530] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10378.230744] LustreError: Skipped 1 previous similar message [10378.252890] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10378.277181] Lustre: Skipped 1 previous similar message [10381.961773] Lustre: server umount lustre-OST0000 complete [10385.890840] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10385.899215] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10385.907408] Lustre: Skipped 1 previous similar message [10385.911307] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10388.451101] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10388.458495] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10388.484902] Lustre: Skipped 1 previous similar message [10393.571782] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10393.579685] Lustre: Skipped 1 previous similar message [10398.688400] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10398.705387] Lustre: Skipped 1 previous similar message [10400.223135] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10400.437318] Lustre: server umount lustre-MDT0000 complete [10405.055390] LustreError: 298640:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786410431 with bad export cookie 7975812888258143382 [10405.059383] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10405.067450] LustreError: 298640:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10405.552954] Lustre: server umount lustre-MDT0001 complete [10416.134142] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10421.509177] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [10426.873409] Lustre: server umount MGS complete [10439.528424] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10440.081371] LustreError: 304635:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10440.095065] LustreError: 304635:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10440.140462] LustreError: 304648:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10440.325824] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10445.282812] LustreError: 304636:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10445.845915] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [10450.407302] LustreError: 304635:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10455.538791] LustreError: 304636:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10455.753973] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10456.194312] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10461.281572] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [10471.401445] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10478.981925] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10488.261244] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10488.674365] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10496.296674] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [10504.758109] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10544.100581] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10544.116882] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10544.137621] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10546.154080] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10547.685781] Lustre: server umount lustre-OST0000 complete [10553.313790] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10553.326630] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10553.349260] Lustre: Skipped 1 previous similar message [10557.855939] Lustre: server umount lustre-MDT0000 complete [10561.509628] LustreError: 304630:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10561.539348] LustreError: 304630:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10562.783247] LustreError: 304616:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786410589 with bad export cookie 7975812888258145321 [10562.793933] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10563.090617] Lustre: server umount lustre-MDT0001 complete [10580.487621] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [10584.149173] Key type lgssc unregistered [10584.690587] LNet: 308587:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10584.701184] LNetError: 308587:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10584.720578] LNet: Removed LNI 192.168.204.135@tcp [10586.323749] Key type .llcrypt unregistered [10586.325984] Key type ._llcrypt unregistered [10618.143787] Key type ._llcrypt registered [10618.148192] Key type .llcrypt registered [10618.315710] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_hostid [10638.166390] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [10640.046082] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10640.155960] alg: No test for adler32 (adler32-zlib) [10641.320978] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [10641.549129] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [10643.271211] Key type lgssc registered [10644.591751] Lustre: Echo OBD driver; http://www.lustre.org/ [10671.031517] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [10688.053474] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10688.100301] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10689.468558] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10689.556611] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10689.676565] Lustre: lustre-MDT0000: new disk, initializing [10689.751720] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10689.772212] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10695.945994] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [10708.513931] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10708.867455] Lustre: lustre-OST0000: new disk, initializing [10708.873327] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10708.878108] Lustre: Skipped 1 previous similar message [10708.887975] Lustre: 312641:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10708.964283] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10716.407661] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [10717.926414] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10717.937633] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10717.984054] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [11680.740608] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11680.755260] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11683.915041] Lustre: server umount lustre-MDT0000 complete [11687.599392] LustreError: 311828:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786411714 with bad export cookie 15858969372441500962 [11687.615953] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11687.816480] Lustre: server umount lustre-OST0000 complete [11716.223186] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_hostid [11728.938172] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [11771.602960] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11771.907416] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11771.937172] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11771.994699] Lustre: lustre-MDT0000: new disk, initializing [11772.046189] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11772.064557] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11778.193357] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [11789.970903] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11790.248701] Lustre: 316996:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11790.366239] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11790.374171] Lustre: Skipped 1 previous similar message [11790.455784] Lustre: lustre-MDT0001: new disk, initializing [11790.648216] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11790.708114] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11790.723393] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11797.539840] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [11808.059542] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11814.329260] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11822.862526] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11823.332603] Lustre: lustre-OST0000: new disk, initializing [11823.344378] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11823.362172] Lustre: 318964:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11823.567059] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11828.844494] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11828.871904] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11829.032054] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11831.940201] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [11842.719405] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11847.862504] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11848.123517] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11852.784724] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11853.050420] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11859.944337] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11859.944685] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11859.955068] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11859.970130] Lustre: Skipped 1 previous similar message [11861.201499] Lustre: server umount lustre-OST0000 complete [11867.617928] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11867.623113] LustreError: Skipped 1 previous similar message [11867.628441] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11867.642673] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11867.651928] Lustre: Skipped 1 previous similar message [11870.177488] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11870.178450] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11871.786649] Lustre: server umount lustre-MDT0000 complete [11875.426786] LustreError: 316988:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786411902 with bad export cookie 15858969372444441781 [11875.429132] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11875.436920] LustreError: 316988:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11875.804624] Lustre: server umount lustre-MDT0001 complete [11879.636933] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11881.807994] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 21:31:48 (1786411908) [11889.539757] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11889.902219] LustreError: 321281:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11889.920225] LustreError: 321281:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11889.988956] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11895.062202] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [11895.275990] LustreError: 321282:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11899.365884] LustreError: 321281:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11903.700363] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11904.073700] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11909.037428] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [11918.838701] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11925.502883] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11932.231888] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11932.823957] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11940.760842] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [11948.336298] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11955.982741] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11956.369063] Lustre: lustre-OST0001: new disk, initializing [11956.376380] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11956.383959] Lustre: 324116:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11956.508072] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11964.276118] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [11965.611574] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11965.621917] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11965.652516] LustreError: 324546:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11965.660331] LustreError: 324546:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11966.692335] LustreError: 324541:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11966.708123] LustreError: 324541:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11967.726309] LustreError: 324546:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11969.831515] LustreError: 324541:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11969.840310] LustreError: 324541:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 4 previous similar messages [11973.931722] LustreError: 324546:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11973.960698] LustreError: 324546:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [11974.757286] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11975.145352] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11981.020908] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff940946251800.ost_server_uuid 50 [11982.183876] LustreError: 324541:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11982.196518] LustreError: 324541:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 16 previous similar messages [11983.217475] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff940946251800.ost_server_uuid in FULL state after 0 sec [11989.866063] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff940946251800.ost_server_uuid 50 [11991.721527] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff940946251800.ost_server_uuid in IDLE state after 0 sec [11993.478437] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [12008.931334] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12008.937629] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12008.947442] LustreError: Skipped 1 previous similar message [12008.951176] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12008.951184] Lustre: Skipped 1 previous similar message [12008.989062] Lustre: Skipped 1 previous similar message [12017.129091] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12017.136086] Lustre: Skipped 6 previous similar messages [12022.243797] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12022.343776] Lustre: server umount lustre-OST0001 complete [12025.867234] LustreError: 323102:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.204.35@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12032.498311] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12032.509492] LustreError: 324549:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12032.516765] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12032.605918] LustreError: 324549:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [12037.225992] Lustre: server umount lustre-OST0000 complete [12042.725868] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12042.733183] Lustre: Skipped 5 previous similar messages [12047.331052] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12048.760928] Lustre: server umount lustre-MDT0000 complete [12052.963001] LustreError: 321281:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12052.980929] LustreError: 321281:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [12053.641083] LustreError: 321262:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786412080 with bad export cookie 15858969372444442663 [12053.647584] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12053.662107] LustreError: 321262:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12054.142568] Lustre: server umount lustre-MDT0001 complete [12065.111494] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [12068.529379] Key type lgssc unregistered [12068.902485] LNet: 326806:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12068.916126] LNetError: 326806:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [12068.929829] LNet: Removed LNI 192.168.204.135@tcp [12069.881273] Key type .llcrypt unregistered [12069.884363] Key type ._llcrypt unregistered [12099.263653] Key type ._llcrypt registered [12099.266130] Key type .llcrypt registered [12099.521267] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [12102.694911] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12102.877530] alg: No test for adler32 (adler32-zlib) [12104.226190] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [12104.646730] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [12106.303984] Key type lgssc registered [12107.628918] Lustre: Echo OBD driver; http://www.lustre.org/ [12117.672888] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12117.690924] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12119.119088] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12126.231501] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [12129.260383] LustreError: 328230:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12129.280934] LustreError: 328230:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [12134.373859] LustreError: 328229:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12135.863417] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12139.490996] LustreError: 328230:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12144.457353] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12144.614198] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [12144.770330] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12150.723329] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [12162.472168] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12163.286832] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12164.263715] LustreError: 329908:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12164.283280] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [12169.388774] LustreError: 329908:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12169.403534] LustreError: 329908:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [12173.260357] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [12179.963436] LustreError: 329959:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12179.994511] LustreError: 329959:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [12182.967436] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12192.896331] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12193.248568] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12198.403555] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [12201.710327] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [12210.969266] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [12219.151944] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [12219.503148] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12326.388549] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12326.397461] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12326.410942] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12326.451726] LustreError: Skipped 1 previous similar message [12329.682898] Lustre: server umount lustre-OST0001 complete [12331.496796] LustreError: 329910:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12331.525036] LustreError: 329910:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [12336.618699] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12336.624407] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12336.642048] Lustre: Skipped 2 previous similar messages [12336.664792] Lustre: Skipped 2 previous similar messages [12341.744391] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12341.757191] Lustre: Skipped 1 previous similar message [12342.555944] Lustre: server umount lustre-OST0000 complete [12349.415978] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12349.439177] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12349.479474] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12351.970702] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12354.047477] Lustre: server umount lustre-MDT0000 complete [12358.820164] LustreError: 331917:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786412385 with bad export cookie 4636609422952622269 [12358.834121] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12358.838118] LustreError: 331917:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12359.371529] Lustre: server umount lustre-MDT0001 complete [12374.615463] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [12377.960791] Key type lgssc unregistered [12378.221714] LNet: 333179:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12378.239664] LNetError: 333179:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [12378.259878] LNet: Removed LNI 192.168.204.135@tcp [12379.180090] Key type .llcrypt unregistered [12379.184156] Key type ._llcrypt unregistered [12436.825639] Key type ._llcrypt registered [12436.827953] Key type .llcrypt registered [12436.948484] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [12438.301983] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12438.334338] alg: No test for adler32 (adler32-zlib) [12439.521750] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [12439.868093] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [12441.591182] Key type lgssc registered [12442.645595] Lustre: Echo OBD driver; http://www.lustre.org/ [12452.990919] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12453.016234] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12454.464144] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12458.963358] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [12464.622265] LustreError: 335472:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12464.665120] LustreError: 335472:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [12468.358337] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12468.632654] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12474.732373] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [12478.174764] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12485.044150] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12485.389465] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12492.715797] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [12493.560285] LustreError: 336962:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12498.940521] LustreError: 336963:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12499.981952] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12502.491965] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12502.703609] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12508.162922] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12509.719454] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [12523.021463] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12532.313721] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12549.096181] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12549.103159] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12549.105188] Lustre: Skipped 2 previous similar messages [12549.120536] Lustre: Skipped 3 previous similar messages [12554.208173] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12554.216480] Lustre: Skipped 3 previous similar messages [12555.274673] Lustre: server umount lustre-MDT0000 complete [12559.354522] LustreError: 335473:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12559.407994] LustreError: 335473:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [12560.415573] LustreError: 338484:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786412587 with bad export cookie 2721526422709242327 [12560.424894] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12560.432592] LustreError: 338484:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12561.057838] Lustre: server umount lustre-MDT0001 complete [12576.414978] Lustre: server umount lustre-OST0000 complete [12580.324791] Lustre: 334735:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786412591/real 1786412591] req@ffff988e45e39c00 x1873189238017920/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786412607 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12580.411146] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12580.456806] Lustre: Skipped 1 previous similar message [12581.503233] Lustre: server umount lustre-OST0001 complete [12601.675275] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [12615.954719] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12616.238377] LustreError: 340740:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12616.321795] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12623.065036] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [12626.916693] LustreError: 340741:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12626.932407] LustreError: 340741:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [12632.750958] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12633.183488] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12638.171079] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [12641.503870] Lustre: 341882:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12649.372451] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12649.803303] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12657.183628] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [12658.035846] LustreError: 342237:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12658.085491] LustreError: 342237:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [12661.143822] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12668.643347] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12674.111678] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12677.114540] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [12685.843663] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12690.830206] Lustre: 343755:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12699.621201] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12699.621936] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12699.627280] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12699.630997] Lustre: Skipped 1 previous similar message [12701.561846] Lustre: server umount lustre-MDT0000 complete [12704.768563] LustreError: 340740:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12704.800945] LustreError: 340740:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [12706.326467] LustreError: 341482:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786412733 with bad export cookie 2721526422709244028 [12706.344731] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12706.657957] Lustre: server umount lustre-MDT0001 complete [12720.273362] Lustre: server umount lustre-OST0000 complete [12734.797445] Lustre: server umount lustre-OST0001 complete [12740.118568] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12742.457961] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12744.699327] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 21:46:11 (1786412771) [12771.089663] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [12780.711828] loop0: detected capacity change from 0 to 1099511627776 [12781.237081] SGI XFS with ACLs, security attributes, quota, no debug enabled [12781.242798] *** VALIDATE xfs *** [12781.250556] XFS (loop0): Mounting V5 Filesystem [12781.275198] XFS (loop0): Ending clean mount [12782.718187] loop1: detected capacity change from 0 to 18691697672192 [12869.686396] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_hostid [12881.332721] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [12928.196515] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12928.704148] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12928.782163] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12928.865695] Lustre: lustre-MDT0000: new disk, initializing [12928.966398] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12928.977751] Lustre: Skipped 1 previous similar message [12929.013366] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12935.328772] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [12948.240491] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12948.432979] Lustre: 348888:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12948.539097] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12948.543252] Lustre: Skipped 1 previous similar message [12948.630306] Lustre: lustre-MDT0001: new disk, initializing [12948.852621] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12948.878524] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12955.063764] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [12966.540344] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12972.173797] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12979.431421] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12979.816167] Lustre: lustre-OST0000: new disk, initializing [12979.822175] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12979.831953] Lustre: 350858:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12981.277395] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12981.284348] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12981.445142] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12987.469371] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [12998.123533] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [13004.432744] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [13004.717900] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [13010.519142] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [13010.842495] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [13017.059051] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13017.071532] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13017.086217] Lustre: Skipped 2 previous similar messages [13017.096426] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13017.101850] Lustre: Skipped 3 previous similar messages [13022.697944] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13022.710198] Lustre: Skipped 2 previous similar messages [13027.295991] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13027.559257] Lustre: server umount lustre-OST0000 complete [13032.930516] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13032.946589] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13032.948406] Lustre: Skipped 2 previous similar messages [13032.956215] Lustre: Skipped 1 previous similar message [13039.030355] Lustre: server umount lustre-MDT0000 complete [13043.179195] LustreError: 348895:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13043.211598] LustreError: 348895:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [13043.695232] LustreError: 348881:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786413070 with bad export cookie 2721526422709245582 [13043.698429] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13043.708840] LustreError: 348881:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [13044.243046] Lustre: server umount lustre-MDT0001 complete [13053.819229] XFS (loop0): Unmounting Filesystem [13059.157273] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 21:51:25 (1786413085) [13067.111559] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13067.462582] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13067.469955] Lustre: Skipped 2 previous similar messages [13073.527910] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [13082.239865] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13087.473913] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [13096.276648] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [13102.239545] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [13109.455217] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13118.796789] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [13128.710971] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [13138.917692] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [13138.923495] LustreError: Skipped 1 previous similar message [13138.927903] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13138.945143] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13138.951242] Lustre: Skipped 2 previous similar messages [13143.024490] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13145.173885] Lustre: server umount lustre-OST0000 complete [13156.255331] Lustre: server umount lustre-MDT0000 complete [13160.561480] LustreError: 353402:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786413187 with bad export cookie 2721526422709246457 [13160.571430] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13160.572201] LustreError: 353402:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [13160.915724] Lustre: server umount lustre-MDT0001 complete [13175.405869] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [13179.451132] Key type lgssc unregistered [13179.768802] LNet: 357050:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13179.784968] LNetError: 357050:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [13180.837733] LNet: Removed LNI 192.168.204.135@tcp [13182.098152] Key type .llcrypt unregistered [13182.101223] Key type ._llcrypt unregistered [13220.450240] Key type ._llcrypt registered [13220.458088] Key type .llcrypt registered [13220.706713] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [13222.371116] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13222.560335] alg: No test for adler32 (adler32-zlib) [13224.120665] Lustre: Lustre: Build Version: 2.17.54_107_gf473267 [13224.585947] LNet: Added LNI 192.168.204.135@tcp [8/256/0/180] [13226.383300] Key type lgssc registered [13227.966531] Lustre: Echo OBD driver; http://www.lustre.org/ [13238.546421] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13238.559813] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13239.948984] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13246.087160] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [13249.525478] LustreError: 358619:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13249.558228] LustreError: 358619:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [13254.627811] LustreError: 358618:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13256.620885] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13257.213711] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13263.857675] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [13273.277828] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [13281.888863] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [13290.382754] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13290.821624] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13298.705499] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [13308.530896] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [13318.625129] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13318.632957] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13318.647958] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13319.658350] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13319.659765] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13320.704920] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.35@tcp (stopping) [13320.710546] Lustre: Skipped 1 previous similar message [13323.157365] Lustre: server umount lustre-MDT0001 complete [13324.769804] LustreError: 360457:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13324.823437] LustreError: 360457:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [13329.897024] LustreError: 358619:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13329.937500] LustreError: 358619:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [13335.016955] LustreError: 358618:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13335.073134] LustreError: 358618:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [13337.076256] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13337.251210] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13337.255624] Lustre: Found index 1 for lustre-MDT0001, updating log [13337.291831] Lustre: 358606:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13337.710979] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13343.215520] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13343.523404] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [13344.252370] LustreError: 362098:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13344.269498] LustreError: 362098:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13344.287966] LustreError: 358611:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13344.382967] LustreError: 362103:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.135@tcp: cfg command failed: rc = -17 [13344.395143] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [13344.395143] [13356.560727] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.35@tcp (stopping) [13358.569704] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13358.582573] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13358.610095] Lustre: Skipped 1 previous similar message [13361.645694] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.35@tcp (stopping) [13361.655612] Lustre: Skipped 2 previous similar messages [13361.901457] Lustre: server umount lustre-MDT0001 complete [13363.705528] LustreError: 358619:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13363.738321] LustreError: 358619:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [13376.049770] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13376.191401] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13376.205541] Lustre: Found index 1 for lustre-MDT0001, updating log [13376.287526] Lustre: 358607:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13376.658319] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13382.124768] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13382.458645] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [13385.219111] LustreError: 363263:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13385.232283] LustreError: 363263:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13385.247832] LustreError: 358611:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13385.265632] LustreError: 358611:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13385.365521] LustreError: 363268:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13385.372994] LustreError: 363268:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13385.388734] LustreError: 363268:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.135@tcp: cfg command failed: rc = -17 [13385.401360] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.135@tcp [13385.401360] [13694.445781] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13694.450476] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13694.464597] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.35@tcp (stopping) [13694.482405] Lustre: Skipped 2 previous similar messages [13696.344413] Lustre: server umount lustre-MDT0001 complete [13699.574684] LustreError: 358614:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.35@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13699.632525] LustreError: 358614:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [13708.147684] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13708.295935] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13708.307090] Lustre: Found index 1 for lustre-MDT0001, updating log [13708.377879] Lustre: 358606:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13708.846869] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13713.907309] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13714.247266] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [13715.458610] LustreError: 371426:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13715.476208] LustreError: 371426:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13715.495891] LustreError: 358611:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13715.515880] LustreError: 358611:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13715.689184] LustreError: 371436:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13715.739784] LustreError: 371436:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.135@tcp: cfg command failed: rc = -114 [13715.757112] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13715.757112] [14036.452288] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14036.458072] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14036.465155] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14036.472543] Lustre: Skipped 3 previous similar messages [14036.481778] Lustre: Skipped 1 previous similar message [14042.402322] Lustre: server umount lustre-MDT0000 complete [14046.049754] LustreError: 358601:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786414072 with bad export cookie 16044848930984925149 [14046.064924] LustreError: 358601:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [14046.067578] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14046.691858] LustreError: 358615:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14046.693653] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14046.707194] LustreError: 358615:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [14052.636641] Lustre: server umount lustre-MDT0001 complete [14056.677268] Lustre: server umount lustre-OST0000 complete [14067.330275] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_hostid [14081.136556] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [14123.041332] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 22:09:09 (1786414149) [14132.648740] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14132.861404] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14132.885473] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14133.042152] Lustre: lustre-MDT0000: new disk, initializing [14133.311429] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14133.366153] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14139.152259] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [14152.278263] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14152.417978] Lustre: 381818:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14152.510896] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14152.519622] Lustre: Skipped 1 previous similar message [14152.628663] Lustre: lustre-MDT0001: new disk, initializing [14152.854666] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14152.939575] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14152.962147] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14158.456582] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [14169.107628] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [14176.474230] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [14185.221589] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14185.515599] Lustre: lustre-OST0000: new disk, initializing [14185.522137] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14185.532473] Lustre: 383790:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14185.690571] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14194.731298] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_default_debug -1 all [14195.780017] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14195.805554] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14195.953253] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14205.711695] Lustre: DEBUG MARKER: oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [14215.151539] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [14215.163526] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14215.179014] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14215.191527] Lustre: Skipped 6 previous similar messages [14216.668537] Lustre: server umount lustre-MDT0000 complete [14216.692653] LustreError: 381825:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14216.742768] LustreError: 381825:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [14222.052415] LustreError: 381812:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786414248 with bad export cookie 16044848930984927102 [14222.067773] LustreError: MGC192.168.204.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14222.585246] Lustre: server umount lustre-MDT0001 complete [14238.231913] Lustre: server umount lustre-OST0000 complete [14275.329763] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing set_hostid [14288.143755] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing load_modules_local [14329.154783] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 14013 sec ========= 22:12:34 (1786414354) [14331.498497] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 22:12:37 (1786414357) === [14337.465658] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 22:12:43 (1786414363) === [14398.473270] Lustre: DEBUG MARKER: oleg435-server.virtnet: executing unload_modules_local [14403.256257] Key type lgssc unregistered [14403.897308] LNet: 390182:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14403.916180] LNetError: 390182:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [14403.943496] LNet: Removed LNI 192.168.204.135@tcp [14405.393217] Key type .llcrypt unregistered [14405.395238] Key type ._llcrypt unregistered