[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 476272634 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524588K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.003249] x2apic enabled [ 0.004011] Switched APIC routing to physical x2apic. [ 0.005014] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009016] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010013] pid_max: default: 32768 minimum: 301 [ 0.011152] LSM: Security Framework initializing [ 0.012051] Yama: becoming mindful. [ 0.013042] SELinux: Initializing. [ 0.015078] *** VALIDATE selinux *** [ 0.023593] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028512] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029171] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030095] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031112] *** VALIDATE tmpfs *** [ 0.032525] *** VALIDATE proc *** [ 0.033197] *** VALIDATE cgroup *** [ 0.034008] *** VALIDATE cgroup2 *** [ 0.035225] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036176] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037010] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038024] Spectre V2 : User space: Vulnerable [ 0.039009] Speculative Store Bypass: Vulnerable [ 0.041846] debug: unmapping init [mem 0xffffffff9f459000-0xffffffff9f460fff] [ 0.043163] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044747] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045027] ... version: 2 [ 0.046013] ... bit width: 48 [ 0.047013] ... generic registers: 4 [ 0.048013] ... value mask: 0000ffffffffffff [ 0.049013] ... max period: 00007fffffffffff [ 0.050012] ... fixed-purpose events: 3 [ 0.051008] ... event mask: 000000070000000f [ 0.052321] rcu: Hierarchical SRCU implementation. [ 0.054507] smp: Bringing up secondary CPUs ... [ 0.055503] x86: Booting SMP configuration: [ 0.056024] .... node #0, CPUs: #1 #2 #3 [ 0.060474] smp: Brought up 1 node, 4 CPUs [ 0.062015] smpboot: Max logical packages: 1 [ 0.063020] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.141097] node 0 deferred pages initialised in 75ms [ 0.144124] devtmpfs: initialized [ 0.145232] x86/mm: Memory block size: 128MB [ 0.149100] gcov: version magic: 0x41383552 [ 0.152298] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.155135] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.159899] pinctrl core: initialized pinctrl subsystem [ 0.162202] [ 0.162653] ************************************************************* [ 0.165014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.168010] ** ** [ 0.171016] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.173012] ** ** [ 0.176013] ** This means that this kernel is built to expose internal ** [ 0.179015] ** IOMMU data structures, which may compromise security on ** [ 0.182012] ** your system. ** [ 0.184013] ** ** [ 0.187012] ** If you see this message and you are not debugging the ** [ 0.191015] ** kernel, report this immediately to your vendor! ** [ 0.193011] ** ** [ 0.196014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.198012] ************************************************************* [ 0.201736] NET: Registered protocol family 16 [ 0.203453] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.206062] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.209062] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.213130] cpuidle: using governor menu [ 0.214906] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.217519] PCI: Using configuration type 1 for base access [ 0.219127] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.228194] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.230034] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.234092] cryptd: max_cpu_qlen set to 1000 [ 0.237818] ACPI: Added _OSI(Module Device) [ 0.238015] ACPI: Added _OSI(Processor Device) [ 0.239048] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.240016] ACPI: Added _OSI(Processor Aggregator Device) [ 0.245732] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.253537] ACPI: Interpreter enabled [ 0.255065] ACPI: PM: (supports S0 S3 S4 S5) [ 0.256012] ACPI: Using IOAPIC for interrupt routing [ 0.258123] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.262465] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.280306] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.282052] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.285022] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.288095] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.293511] acpiphp: Slot [2] registered [ 0.295115] acpiphp: Slot [5] registered [ 0.296126] acpiphp: Slot [6] registered [ 0.297122] acpiphp: Slot [7] registered [ 0.299126] acpiphp: Slot [8] registered [ 0.300142] acpiphp: Slot [9] registered [ 0.302116] acpiphp: Slot [10] registered [ 0.304100] acpiphp: Slot [3] registered [ 0.305096] acpiphp: Slot [4] registered [ 0.306120] acpiphp: Slot [11] registered [ 0.308149] acpiphp: Slot [12] registered [ 0.310118] acpiphp: Slot [13] registered [ 0.312116] acpiphp: Slot [14] registered [ 0.314127] acpiphp: Slot [15] registered [ 0.315179] acpiphp: Slot [16] registered [ 0.317196] acpiphp: Slot [17] registered [ 0.318111] acpiphp: Slot [18] registered [ 0.319139] acpiphp: Slot [19] registered [ 0.321128] acpiphp: Slot [20] registered [ 0.323175] acpiphp: Slot [21] registered [ 0.324114] acpiphp: Slot [22] registered [ 0.326110] acpiphp: Slot [23] registered [ 0.328097] acpiphp: Slot [24] registered [ 0.329103] acpiphp: Slot [25] registered [ 0.331102] acpiphp: Slot [26] registered [ 0.332115] acpiphp: Slot [27] registered [ 0.333143] acpiphp: Slot [28] registered [ 0.334094] acpiphp: Slot [29] registered [ 0.335083] acpiphp: Slot [30] registered [ 0.336094] acpiphp: Slot [31] registered [ 0.338054] PCI host bridge to bus 0000:00 [ 0.339017] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.341025] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.343022] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.345021] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.348023] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.350025] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.352157] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.355025] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.358322] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.368829] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.373554] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.376020] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.379020] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.381018] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.384490] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.387687] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.390044] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.393692] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.398011] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.409015] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.416017] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.422079] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.434020] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.445016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.471018] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.480439] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.486018] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.494018] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.515019] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.530665] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.542019] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.552021] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.566021] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.578241] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.590021] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.598021] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.615015] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.626095] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.633017] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.641021] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.657018] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.667954] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.683016] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.688015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.703028] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.718753] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.720330] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.722226] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.724261] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.725131] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.728292] iommu: Default domain type: Passthrough [ 0.729311] SCSI subsystem initialized [ 0.730069] ACPI: bus type USB registered [ 0.731084] usbcore: registered new interface driver usbfs [ 0.732042] usbcore: registered new interface driver hub [ 0.733069] usbcore: registered new device driver usb [ 0.734131] pps_core: LinuxPPS API ver. 1 registered [ 0.736010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.737047] PTP clock support registered [ 0.739093] EDAC MC: Ver: 3.0.0 [ 0.740326] PCI: Using ACPI for IRQ routing [ 0.741383] NetLabel: Initializing [ 0.742008] NetLabel: domain hash size = 128 [ 0.742853] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.744077] NetLabel: unlabeled traffic allowed by default [ 0.746103] vgaarb: loaded [ 0.747201] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.748011] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.755000] clocksource: Switched to clocksource kvm-clock [ 0.860773] VFS: Disk quotas dquot_6.6.0 [ 0.862335] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.865070] *** VALIDATE ramfs *** [ 0.866914] *** VALIDATE hugetlbfs *** [ 0.869397] pnp: PnP ACPI init [ 0.871654] pnp: PnP ACPI: found 6 devices [ 0.887551] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.890564] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.892487] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.894475] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.896719] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.899149] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.902044] NET: Registered protocol family 2 [ 0.904609] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.909042] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.912854] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.918095] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.921147] TCP: Hash tables configured (established 65536 bind 65536) [ 0.923534] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.926171] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.929359] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.932460] NET: Registered protocol family 1 [ 0.936628] RPC: Registered named UNIX socket transport module. [ 0.938692] RPC: Registered udp transport module. [ 0.940471] RPC: Registered tcp transport module. [ 0.942093] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.944510] NET: Registered protocol family 44 [ 0.945910] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.948963] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.951051] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.952981] PCI: CLS 0 bytes, default 64 [ 0.954449] Unpacking initramfs... [ 2.345975] debug: unmapping init [mem 0xffff9444bcc54000-0xffff9444bffbffff] [ 2.349934] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.352013] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.354662] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.872592] Initialise system trusted keyrings [ 2.873904] Key type blacklist registered [ 2.875372] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.885656] zbud: loaded [ 2.889118] *** VALIDATE nfs *** [ 2.890134] *** VALIDATE nfs4 *** [ 2.891324] pstore: using deflate compression [ 2.894023] Platform Keyring initialized [ 2.977572] NET: Registered protocol family 38 [ 2.979304] Key type asymmetric registered [ 2.980831] Asymmetric key parser 'x509' registered [ 2.982705] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.985980] io scheduler mq-deadline registered [ 2.987847] io scheduler kyber registered [ 2.989600] io scheduler bfq registered [ 2.991970] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.995269] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.997727] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.001082] ACPI: Power Button [PWRF] [ 3.007092] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.014107] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.038802] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.044600] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.059935] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.091291] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.119320] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.124792] Non-volatile memory driver v1.3 [ 3.126956] Linux agpgart interface v0.103 [ 3.164306] virtio_blk virtio1: [vda] 146000 512-byte logical blocks (74.8 MB/71.3 MiB) [ 3.167357] vda: detected capacity change from 0 to 74752000 [ 3.181521] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.184645] vdb: detected capacity change from 0 to 1073741824 [ 3.198549] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.201084] vdc: detected capacity change from 0 to 2621440000 [ 3.215879] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.218741] vdd: detected capacity change from 0 to 2621440000 [ 3.231583] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.234589] vde: detected capacity change from 0 to 4294967296 [ 3.289178] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.291972] vdf: detected capacity change from 0 to 4294967296 [ 3.298176] libphy: Fixed MDIO Bus: probed [ 3.303431] usbcore: registered new interface driver usbserial_generic [ 3.305961] usbserial: USB Serial support registered for generic [ 3.308256] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.313906] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.316080] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.318432] mousedev: PS/2 mouse device common for all mice [ 3.321329] rtc_cmos 00:05: RTC can wake from S4 [ 3.324967] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.328085] rtc_cmos 00:05: registered as rtc0 [ 3.333956] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.336945] intel_pstate: CPU model not supported [ 3.340986] hid: raw HID events driver (C) Jiri Kosina [ 3.343115] usbcore: registered new interface driver usbhid [ 3.345877] usbhid: USB HID core driver [ 3.347965] drop_monitor: Initializing network drop monitor service [ 3.348905] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.351609] Initializing XFRM netlink socket [ 3.352462] NET: Registered protocol family 10 [ 3.364846] Segment Routing with IPv6 [ 3.366441] NET: Registered protocol family 17 [ 3.368866] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.368995] mpls_gso: MPLS GSO support [ 3.384154] RAS: Correctable Errors collector initialized. [ 3.386301] AVX version of gcm_enc/dec engaged. [ 3.388050] AES CTR mode by8 optimization enabled [ 3.470561] sched_clock: Marking stable (3470476593, 0)->(4381500548, -911023955) [ 3.474420] registered taskstats version 1 [ 3.476586] Loading compiled-in X.509 certificates [ 3.479145] zswap: loaded using pool lzo/zbud [ 3.507512] Key type big_key registered [ 3.518952] Key type encrypted registered [ 3.521141] ima: No TPM chip found, activating TPM-bypass! [ 3.523556] ima: Allocated hash algorithm: sha1 [ 3.525615] ima: No architecture policies found [ 3.527527] evm: Initialising EVM extended attributes: [ 3.529475] evm: security.selinux [ 3.530764] evm: security.ima [ 3.532171] evm: security.capability [ 3.533545] evm: HMAC attrs: 0x1 [ 3.535703] rtc_cmos 00:05: setting system clock to 2026-08-19 04:50:01 UTC (1787115001) [ 3.543278] debug: unmapping init [mem 0xffffffffa0403000-0xffffffffa05fffff] [ 3.546301] debug: unmapping init [mem 0xffffffff9f182000-0xffffffff9f458fff] [ 3.559136] Write protecting the kernel read-only data: 28672k [ 3.562578] debug: unmapping init [mem 0xffffffff9d803000-0xffffffff9d9fffff] [ 3.565089] debug: unmapping init [mem 0xffffffff9e114000-0xffffffff9e1fffff] [ 3.600886] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.609853] systemd[1]: Detected virtualization kvm. [ 3.611672] systemd[1]: Detected architecture x86-64. [ 3.613784] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.639266] systemd[1]: No hostname configured. [ 3.641375] systemd[1]: Set hostname to . [ 3.643469] random: systemd: uninitialized urandom read (16 bytes read) [ 3.645841] systemd[1]: Initializing machine ID from random generator. [ 3.787595] random: systemd: uninitialized urandom read (16 bytes read) [ 3.790721] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 3.796986] random: systemd: uninitialized urandom read (16 bytes read) [ 3.799761] systemd[1]: Reached target Paths. [ OK ] Reached target Paths. [ 3.803622] systemd[1]: Reached target Local Encrypted Volumes. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Local File Systems. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket. Starting Journal Service... Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Swap. [ OK ] Reached target Slices. [ OK ] Reached target Timers. Starting Create list of required st…ce nodes for the current kernel... Starting Create Volatile Files and Directories... Starting Apply Kernel Variables... [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.417158] device-mapper: uevent: version 1.0.3 [ 4.419550] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.413411] virtio_net virtio0 ens2: renamed from eth0 [ 5.422124] random: fast init done [ 5.433318] scsi host0: ata_piix [ 5.535687] scsi host1: ata_piix [ 5.537561] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.540476] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.039576] random: crng init done [ 10.041100] random: 7 urandom warning(s) missed due to ratelimiting [ 10.045434] dracut-initqueue[586]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.743592] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.978202] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.205294] SELinux: Disabled at runtime. [ 12.268182] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.277922] systemd[1]: Detected virtualization kvm. [ 12.280119] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.795215] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.800174] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.805361] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.809417] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.814258] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.825132] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.847486] systemd[1]: Listening on Process Core Dump Socket. [ OK ] Listening on Process Core Dump Socket. Starting Remount Root and Kernel File Systems... Starting Apply Kernel Variables... [ OK ] Reached target rpc_pipefs.target. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Created slice system-getty.slice. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Control Socket. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... Mounting POSIX Message Queue File System... Mounting Kernel Debug File System... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Stopped target Initrd File Systems. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Reached target Local Encrypted Volumes. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting Huge Pages File System... [ 12.988824] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Reached target Paths. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 13.319760] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.759245] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.789248] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.999699] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 14.016836] EDAC sbridge: Ver: 1.1.2 [ 16.318766] Key type dns_resolver registered [ 17.233738] NFS: Registering the id_resolver key type [ 17.237938] Key type id_resolver registered [ 17.244567] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg255-server login: [ 29.869016] hrtimer: interrupt took 5002128 ns [ 83.904591] libcfs: loading out-of-tree module taints kernel. [ 83.954517] Key type ._llcrypt registered [ 83.958132] Key type .llcrypt registered [ 84.067198] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 104.024728] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 106.124508] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 106.138902] alg: No test for adler32 (adler32-zlib) [ 107.966417] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [ 109.131069] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 110.938095] Key type lgssc registered [ 113.117640] Lustre: Echo OBD driver; http://www.lustre.org/ [ 130.134264] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 177.798257] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 193.150311] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 193.175485] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 194.695377] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 194.777939] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 194.973355] Lustre: lustre-MDT0000: new disk, initializing [ 195.126923] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 195.141495] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 200.516534] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 214.626751] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 214.749805] Lustre: 6514:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 214.787065] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 214.793145] Lustre: Skipped 1 previous similar message [ 214.899462] Lustre: lustre-MDT0001: new disk, initializing [ 214.995372] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 215.027370] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 215.033160] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 220.102583] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 226.171244] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 239.330290] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 239.838472] Lustre: lustre-OST0000: new disk, initializing [ 239.847951] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 239.860458] Lustre: 8451:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 239.992879] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 247.654199] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 248.867079] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 248.886431] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 248.977263] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 262.920854] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 263.050968] Lustre: lustre-OST0001: new disk, initializing [ 263.054141] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 263.057539] Lustre: 9525:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 263.119525] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 269.776354] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 271.462246] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 271.493259] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 271.605388] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 282.583362] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 289.688309] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 297.447156] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing check_logdir /tmp/testlogs/ [ 303.114667] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing yml_node [ 307.220107] Lustre: DEBUG MARKER: Client: 2.17.56.3 [ 310.795716] Lustre: DEBUG MARKER: MDS: 2.17.56.3 [ 313.865920] Lustre: DEBUG MARKER: OSS: 2.17.56.3 [ 315.235916] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Aug 19 00:55:12 EDT 2026 [ 334.639909] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 336.495613] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 353.251420] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 353.266971] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 353.301725] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 353.784053] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 353.804432] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 353.812210] Lustre: Skipped 2 previous similar messages [ 358.435716] Lustre: server umount lustre-MDT0000 complete [ 363.175365] LustreError: 10398:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787115361 with bad export cookie 13280913214374672342 [ 363.180612] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 363.184799] LustreError: 10398:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 364.000591] LustreError: 6521:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 364.002820] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 364.005115] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 364.026559] LustreError: 6521:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 364.074559] Lustre: Skipped 3 previous similar messages [ 369.120730] LustreError: 6520:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 369.120988] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 369.161292] LustreError: 6520:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 369.205600] Lustre: Skipped 2 previous similar messages [ 369.786657] Lustre: server umount lustre-MDT0001 complete [ 374.233177] Lustre: server umount lustre-OST0000 complete [ 378.195994] Lustre: server umount lustre-OST0001 complete [ 385.316918] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 396.165980] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 432.243406] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 432.451794] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 432.475179] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 432.593296] Lustre: lustre-MDT0000: new disk, initializing [ 432.690255] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 432.715322] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 437.193541] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 448.639906] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 448.713745] Lustre: 15078:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 448.759519] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 448.762901] Lustre: Skipped 1 previous similar message [ 448.888737] Lustre: lustre-MDT0001: new disk, initializing [ 448.994524] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 449.036222] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 449.049526] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 452.802652] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 462.263937] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 467.784621] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 474.004557] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 474.371516] Lustre: lustre-OST0000: new disk, initializing [ 474.375469] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 474.384118] Lustre: 17049:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 474.507454] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 476.000516] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 476.011215] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 476.095643] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 481.132621] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 490.569694] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 494.742723] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 494.956694] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 499.694426] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 499.919608] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 506.860392] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 506.867255] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 506.886321] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 508.074421] Lustre: server umount lustre-OST0000 complete [ 511.973266] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 511.991290] Lustre: Skipped 1 previous similar message [ 517.091283] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 517.101809] Lustre: Skipped 4 previous similar messages [ 526.303167] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 526.630720] Lustre: server umount lustre-MDT0000 complete [ 530.765769] LustreError: 15071:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787115528 with bad export cookie 13280913214374673875 [ 530.784512] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 531.426411] Lustre: server umount lustre-MDT0001 complete [ 537.091766] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 00:58:54 (1787115534) [ 544.980990] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 545.446825] LustreError: 19313:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 545.524211] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 550.201365] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 550.885199] LustreError: 19314:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 556.011784] LustreError: 19313:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 559.120045] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 564.275915] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 571.778881] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 576.992454] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 583.921391] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 590.716695] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 599.224361] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 603.085587] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 610.793059] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 610.793173] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 610.799170] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 610.799179] Lustre: Skipped 4 previous similar messages [ 610.807237] LustreError: Skipped 1 previous similar message [ 610.831937] Lustre: Skipped 3 previous similar messages [ 616.635641] Lustre: server umount lustre-MDT0000 complete [ 620.790553] LustreError: 19294:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787115618 with bad export cookie 13280913214374674757 [ 620.795371] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 620.804515] LustreError: 19294:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 621.025399] LustreError: 19309:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 621.224615] Lustre: server umount lustre-MDT0001 complete [ 631.195879] Lustre: server umount lustre-OST0000 complete [ 651.722732] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 660.703054] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 661.250331] LustreError: 23696:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 661.354142] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 661.358986] Lustre: Skipped 2 previous similar messages [ 666.673977] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 676.638758] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 682.796668] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 685.696539] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 691.800967] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 698.421793] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 706.845343] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 707.083346] Lustre: lustre-OST0001: new disk, initializing [ 707.086860] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 707.092146] Lustre: 26027:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 709.066961] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 709.075880] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 709.163850] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 712.058139] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 721.917619] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 726.264969] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 739.819101] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 739.832560] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 739.855959] Lustre: Skipped 4 previous similar messages [ 739.881835] Lustre: Skipped 9 previous similar messages [ 745.948977] Lustre: server umount lustre-MDT0000 complete [ 749.513947] LustreError: 23677:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787115747 with bad export cookie 13280913214374675604 [ 749.518171] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 749.529423] LustreError: 23677:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 749.935238] Lustre: server umount lustre-MDT0001 complete [ 764.509863] Lustre: server umount lustre-OST0000 complete [ 766.431195] Lustre: 3647:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787115748/real 1787115748] req@ffff944510fb6300 x1873926010679552/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787115764 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 768.972835] Lustre: server umount lustre-OST0001 complete [ 793.300437] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 804.380246] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 804.768326] LustreError: 29046:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 804.784648] LustreError: 29046:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 804.843316] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 804.848184] Lustre: Skipped 3 previous similar messages [ 809.415503] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 819.144614] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 825.313614] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 829.550339] Lustre: 30194:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 837.593753] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 845.323202] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 854.313462] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 862.731206] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 871.983482] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 876.457803] Lustre: 32074:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 885.734688] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 885.738791] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 885.738803] Lustre: Skipped 4 previous similar messages [ 885.753645] Lustre: Skipped 3 previous similar messages [ 890.000894] Lustre: server umount lustre-MDT0000 complete [ 890.852431] LustreError: 30208:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 890.874887] LustreError: 30208:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 11 previous similar messages [ 894.467044] LustreError: 29027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787115892 with bad export cookie 13280913214374676899 [ 894.470830] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 894.482636] LustreError: 29027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 894.919532] Lustre: server umount lustre-MDT0001 complete [ 909.923316] Lustre: server umount lustre-OST0000 complete [ 911.329583] Lustre: 3647:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787115893/real 1787115893] req@ffff944536edd880 x1873926010731264/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787115909 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 914.803802] Lustre: server umount lustre-OST0001 complete [ 925.679536] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 01:05:22 (1787115922) [ 957.135300] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 968.454942] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 973.879834] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 984.918686] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 990.822812] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 994.636704] Lustre: 36026:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1002.039985] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1009.102547] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1018.231380] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1024.957809] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1033.515376] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1037.522694] Lustre: 37909:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1044.480206] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1044.495907] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1044.511299] Lustre: Skipped 5 previous similar messages [ 1044.525817] Lustre: Skipped 6 previous similar messages [ 1048.749285] Lustre: server umount lustre-MDT0000 complete [ 1049.572623] LustreError: 34878:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1049.594345] LustreError: 34878:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 10 previous similar messages [ 1052.189841] LustreError: 34860:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787116050 with bad export cookie 13280913214374678005 [ 1052.194784] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1052.202596] LustreError: 34860:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1052.422766] Lustre: server umount lustre-MDT0001 complete [ 1066.885544] Lustre: server umount lustre-OST0000 complete [ 1070.047571] Lustre: 3645:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787116052/real 1787116052] req@ffff944403139880 x1873926010781696/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787116068 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1071.635963] Lustre: server umount lustre-OST0001 complete [ 1081.791898] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 01:07:58 (1787116078) [ 1102.075743] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 1113.185633] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1113.837703] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1113.844774] Lustre: Skipped 7 previous similar messages [ 1118.149475] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1128.136067] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1132.675743] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1136.435667] Lustre: 41199:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1143.333957] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1151.491974] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1160.355171] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1167.839655] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1176.630048] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1180.615902] Lustre: 43076:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1181.427276] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1181.434469] Lustre: Skipped 1 previous similar message [ 1184.736602] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1184.742903] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1184.750980] Lustre: Skipped 1 previous similar message [ 1190.580863] Lustre: server umount lustre-MDT0000 complete [ 1194.679207] LustreError: 40032:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787116192 with bad export cookie 13280913214374679111 [ 1194.684694] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1194.689019] LustreError: 40032:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1201.206077] Lustre: server umount lustre-MDT0001 complete [ 1208.972038] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1224.208527] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1230.849257] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1232.600495] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1237.586778] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1238.528364] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1238.537306] Lustre: Skipped 1 previous similar message [ 1245.539824] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1247.300427] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1252.629154] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1254.184863] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1263.586103] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1265.326218] Lustre: server umount lustre-MDT0000 complete [ 1269.046888] LustreError: 40030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787116267 with bad export cookie 13280913214374680238 [ 1269.071104] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1269.755256] Lustre: server umount lustre-MDT0001 complete [ 1275.694540] Lustre: server umount lustre-OST0000 complete [ 1280.112817] Lustre: server umount lustre-OST0001 complete [ 1290.842564] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 01:11:27 (1787116287) [ 1307.336225] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 1318.722426] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1319.275917] LustreError: 47681:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1319.313113] LustreError: 47681:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 28 previous similar messages [ 1324.000623] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1333.422347] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1338.255843] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1341.893695] Lustre: 48843:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1349.760435] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1356.407803] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1365.692553] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1372.295910] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1380.335741] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1385.137374] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1396.629124] Lustre: server umount lustre-MDT0000 complete [ 1400.112546] LustreError: 47662:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787116398 with bad export cookie 13280913214374681029 [ 1400.117661] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1400.124476] LustreError: 47662:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 1400.376325] Lustre: server umount lustre-MDT0001 complete [ 1414.094614] Lustre: server umount lustre-OST0000 complete [ 1417.183210] Lustre: 3644:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787116399/real 1787116399] req@ffff94453c2eb100 x1873926010923392/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787116415 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1418.112353] Lustre: server umount lustre-OST0001 complete [ 1428.471590] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 01:13:45 (1787116425) [ 1432.173534] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1434.274659] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 01:13:51 (1787116431) [ 1445.935794] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 1457.677470] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 1482.789046] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1483.033342] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1483.097775] Lustre: lustre-MDT0000: new disk, initializing [ 1483.185557] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1487.833334] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1498.589802] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1505.430391] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1505.702517] Lustre: lustre-OST0000: new disk, initializing [ 1505.717940] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1505.726949] Lustre: Skipped 1 previous similar message [ 1505.740954] Lustre: 54917:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1507.112795] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1507.117403] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1507.144895] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1511.486407] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1520.621456] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1522.988289] Lustre: DEBUG MARKER: create test files [ 1558.505036] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1558.514219] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1558.531634] Lustre: Skipped 16 previous similar messages [ 1558.541660] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1558.546792] Lustre: Skipped 19 previous similar messages [ 1561.202759] Lustre: server umount lustre-OST0000 complete [ 1565.784469] Lustre: server umount lustre-MDT0000 complete [ 1575.980764] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 1578.598369] Key type lgssc unregistered [ 1579.111736] LNet: 56543:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1579.123459] LNetError: 56543:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1579.149898] LNet: Removed LNI 192.168.202.155@tcp [ 1580.380251] Key type .llcrypt unregistered [ 1580.382447] Key type ._llcrypt unregistered [ 1610.798405] Key type ._llcrypt registered [ 1610.801144] Key type .llcrypt registered [ 1610.941557] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 1612.667349] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1612.742393] alg: No test for adler32 (adler32-zlib) [ 1613.748613] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [ 1614.067674] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 1615.831293] Key type lgssc registered [ 1616.964259] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1624.282680] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1624.324160] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1625.844361] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1630.245799] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1638.108779] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1644.528731] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1644.964974] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1650.096128] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:99 to 0x240000400:129) [ 1650.139310] Lustre: 59281:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1650.145663] Lustre: 59281:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1650.152452] Lustre: 59281:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1650.155362] Lustre: 59281:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1650.162222] Lustre: 59281:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1650.166115] Lustre: 59281:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1651.266950] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1658.411626] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1663.790096] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1669.353188] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1706.481853] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1706.491189] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1706.513840] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1707.822822] Lustre: server umount lustre-OST0000 complete [ 1713.059307] Lustre: server umount lustre-MDT0000 complete [ 1724.736159] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 1727.810981] Key type lgssc unregistered [ 1728.066664] LNet: 60912:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1728.073175] LNetError: 60912:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1728.091071] LNet: Removed LNI 192.168.202.155@tcp [ 1728.789176] Key type .llcrypt unregistered [ 1728.792826] Key type ._llcrypt unregistered [ 1760.588205] Key type ._llcrypt registered [ 1760.590275] Key type .llcrypt registered [ 1760.685031] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 1761.753235] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1761.787624] alg: No test for adler32 (adler32-zlib) [ 1762.865573] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [ 1763.114559] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 1764.863214] Key type lgssc registered [ 1765.709184] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1773.393806] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1773.425421] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1774.976785] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1779.660703] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1786.984985] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1793.826332] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1794.129920] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1799.301166] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1799.394146] Lustre: 63751:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1799.407863] Lustre: 63751:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1799.417172] Lustre: 63751:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1799.423500] Lustre: 63751:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1799.433162] Lustre: 63751:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1799.442618] Lustre: 63751:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1800.334812] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1808.019175] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1812.853068] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1825.252144] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1825.263644] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1825.285091] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1829.811869] Lustre: server umount lustre-OST0000 complete [ 1833.824889] Lustre: server umount lustre-MDT0000 complete [ 1845.287728] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 1847.932728] Key type lgssc unregistered [ 1848.180737] LNet: 65307:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1848.184686] LNetError: 65307:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1848.194366] LNet: Removed LNI 192.168.202.155@tcp [ 1849.090306] Key type .llcrypt unregistered [ 1849.092299] Key type ._llcrypt unregistered [ 1867.678870] Key type ._llcrypt registered [ 1867.681589] Key type .llcrypt registered [ 1867.770909] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 1886.506301] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 1888.145662] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1888.164092] alg: No test for adler32 (adler32-zlib) [ 1889.326721] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [ 1889.607860] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 1891.255184] Key type lgssc registered [ 1892.496689] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1936.169469] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1936.213988] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1937.566036] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1937.619837] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1937.752592] Lustre: lustre-MDT0000: new disk, initializing [ 1937.888609] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1937.930691] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1943.392613] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1955.308422] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1955.419345] Lustre: 68463:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1955.478997] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1955.484170] Lustre: Skipped 1 previous similar message [ 1955.559892] Lustre: lustre-MDT0001: new disk, initializing [ 1955.658076] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1955.704142] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1955.717106] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1960.053755] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 1970.500229] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1977.349607] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1985.405203] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1985.631771] Lustre: lustre-OST0000: new disk, initializing [ 1985.634971] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1985.640532] Lustre: 70433:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1985.717679] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1991.741466] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1991.756590] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1991.943342] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1993.589247] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2004.064680] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2010.510649] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2010.692837] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2017.339346] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2017.555152] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2022.881152] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2022.887655] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2022.895555] LustreError: Skipped 1 previous similar message [ 2022.917040] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2022.930923] Lustre: Skipped 1 previous similar message [ 2026.434637] Lustre: server umount lustre-OST0000 complete [ 2032.612156] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2032.631110] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2032.656143] Lustre: Skipped 1 previous similar message [ 2032.685970] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2037.160984] Lustre: server umount lustre-MDT0000 complete [ 2041.651248] LustreError: 68455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787117039 with bad export cookie 9006421497065588911 [ 2041.667505] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2041.895739] Lustre: server umount lustre-MDT0001 complete [ 2052.432341] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 01:24:09 (1787117049) [ 2065.478221] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 2109.741967] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 2120.537701] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 2161.743792] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2162.005676] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2162.055910] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2162.165925] Lustre: lustre-MDT0000: new disk, initializing [ 2162.279260] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2162.303280] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2167.797256] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2179.409774] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2179.622838] Lustre: 76458:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2179.674712] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2179.678749] Lustre: Skipped 1 previous similar message [ 2179.796350] Lustre: lustre-MDT0001: new disk, initializing [ 2179.966815] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2180.032555] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2180.049786] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2185.504672] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2196.397846] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2202.311657] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2209.132410] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2209.362738] Lustre: lustre-OST0000: new disk, initializing [ 2209.366238] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2209.371596] Lustre: 78425:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2209.446835] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2215.226479] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2216.992054] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2217.002594] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2217.128838] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2223.484870] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2228.782483] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2228.974689] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2233.495861] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2233.733990] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2237.409572] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2237.418522] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2237.433443] Lustre: Skipped 1 previous similar message [ 2237.436290] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2237.444500] Lustre: Skipped 2 previous similar messages [ 2241.767742] Lustre: server umount lustre-OST0000 complete [ 2246.631192] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2246.646740] LustreError: Skipped 1 previous similar message [ 2246.656517] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2246.680099] Lustre: Skipped 1 previous similar message [ 2246.697083] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2246.700492] Lustre: Skipped 1 previous similar message [ 2252.022197] Lustre: server umount lustre-MDT0000 complete [ 2253.280145] LustreError: 76469:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2253.293108] LustreError: 76469:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2255.750240] LustreError: 76450:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787117253 with bad export cookie 9006421497065589800 [ 2255.756491] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2255.765063] LustreError: 76450:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2256.249110] Lustre: server umount lustre-MDT0001 complete [ 2266.208118] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 01:27:43 (1787117263) [ 2275.649603] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2276.539549] LustreError: 80838:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2276.684579] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2281.953682] LustreError: 80839:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2282.293760] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2286.049307] LustreError: 80838:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2291.171077] LustreError: 80839:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2292.421705] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2297.065488] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2306.924474] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2315.151780] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2324.853817] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2325.138265] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2325.144187] Lustre: Skipped 1 previous similar message [ 2331.060124] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2338.555660] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2341.809314] Lustre: 83511:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 170be787-6134-4677-bb01-241db8beef20 at adminstrative request [ 2342.887963] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2342.906594] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2352.170955] Lustre: MGS: Client 170be787-6134-4677-bb01-241db8beef20 (at 0@lo) reconnecting [ 2352.196929] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x7cfd3ca22e094f9a to 0x7cfd3ca22e095233 [ 2352.219323] Lustre: MGC192.168.202.155@tcp: Connection restored to 0@lo (at 0@lo) [ 2380.976341] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2381.288355] Lustre: lustre-OST0001: new disk, initializing [ 2381.292431] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2381.298319] Lustre: 83904:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2388.100788] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2388.132681] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2388.298549] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2389.037810] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2401.005709] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2408.931220] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2408.933336] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2408.938909] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2408.938918] Lustre: Skipped 2 previous similar messages [ 2408.939160] Lustre: Skipped 2 previous similar messages [ 2408.955713] LustreError: Skipped 1 previous similar message [ 2410.152555] Lustre: server umount lustre-OST0001 complete [ 2414.065424] LustreError: 84212:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2414.085574] LustreError: 84212:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2418.656203] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2418.661683] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2418.702743] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2418.709449] Lustre: Skipped 1 previous similar message [ 2421.108883] Lustre: server umount lustre-OST0000 complete [ 2439.649024] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2439.664925] Lustre: Skipped 7 previous similar messages [ 2441.189279] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2441.448847] Lustre: server umount lustre-MDT0000 complete [ 2444.768639] LustreError: 80833:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2444.781770] LustreError: 80833:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2445.752938] LustreError: 80819:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787117443 with bad export cookie 9006421497065591347 [ 2445.754274] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2445.761586] LustreError: 80819:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2446.083976] Lustre: server umount lustre-MDT0001 complete [ 2456.705451] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 01:30:53 (1787117453) [ 2458.543383] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2460.536975] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 01:30:57 (1787117457) [ 2462.279507] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2463.754295] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 01:31:01 (1787117461) [ 2465.320510] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2467.324981] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 01:31:04 (1787117464) [ 2469.215935] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2471.337361] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 01:31:08 (1787117468) [ 2473.589801] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2479.721519] loop: module loaded [ 2479.753851] loop0: detected capacity change from 0 to 4294967296 [ 2479.855779] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2481.494620] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2484.893299] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2489.381983] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2519.311417] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2519.361510] LustreError: 88034:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2519.367557] LustreError: 88034:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2519.372368] LustreError: 88034:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2519.397822] LustreError: 88034:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2541.627673] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 2553.699796] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 2591.705691] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2592.013433] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2592.043646] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2592.128122] Lustre: lustre-MDT0000: new disk, initializing [ 2592.208884] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2592.211973] Lustre: Skipped 1 previous similar message [ 2592.259209] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2596.705337] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2606.912190] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2607.019544] Lustre: 90903:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2607.254818] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2611.702925] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2621.464297] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2629.583441] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2640.141725] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2640.558582] Lustre: 92870:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2647.547476] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2650.748269] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2650.944031] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2659.401596] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2665.668445] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2665.879647] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2671.298320] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2671.494992] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2676.707351] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2676.707674] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2676.726574] LustreError: Skipped 1 previous similar message [ 2676.732703] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2676.744683] Lustre: Skipped 4 previous similar messages [ 2679.773928] Lustre: server umount lustre-OST0000 complete [ 2690.492161] Lustre: server umount lustre-MDT0000 complete [ 2692.064021] LustreError: 90910:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2692.103122] LustreError: 90910:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2694.851657] LustreError: 90897:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787117692 with bad export cookie 9006421497065591970 [ 2694.859165] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2694.861572] LustreError: 90897:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2695.346547] Lustre: server umount lustre-MDT0001 complete [ 2706.752037] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 01:35:03 (1787117703) [ 2715.231464] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2721.833678] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2732.124235] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2732.715564] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2732.730612] Lustre: Skipped 3 previous similar messages [ 2739.045712] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2748.340667] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2754.753508] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2764.936588] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2772.479856] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2783.709495] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2790.538921] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2790.782016] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2797.409632] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2797.635558] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2806.579478] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2806.770685] Lustre: lustre-OST0001: new disk, initializing [ 2806.773166] Lustre: Skipped 2 previous similar messages [ 2806.779465] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2806.784447] Lustre: Skipped 3 previous similar messages [ 2806.788836] Lustre: 98658:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2814.066464] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2816.181176] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2816.194146] Lustre: Skipped 2 previous similar messages [ 2816.209918] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2816.406079] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2825.231119] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2830.149156] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2830.368840] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2835.845977] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2836.023571] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2857.602518] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2883.773406] Lustre: Failing over lustre-MDT0000 [ 2884.307761] Lustre: server umount lustre-MDT0000 complete [ 2884.511700] LustreError: 99287:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.55@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2884.523215] LustreError: 99287:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 2886.119601] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2886.126466] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2886.139506] Lustre: Skipped 2 previous similar messages [ 2904.547298] Lustre: 66685:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787117886/real 1787117886] req@00000000c77063f4 x1873927877190784/t0(0) o400->MGC192.168.202.155@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787117902 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2904.579967] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2911.901258] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2911.904094] LDISKFS-fs (dm-0): recovery complete [ 2911.911129] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2914.145392] LustreError: 101475:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 2914.198295] LustreError: 101475:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@00000000ac46eb59 x1873927877196288/t0(0) o250->MGC192.168.202.155@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1787117912 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2914.235173] LustreError: 101475:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 2915.821184] LustreError: 66684:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000b35c17b0 x1873927877200128/t0(0) o250->MGC192.168.202.155@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2916.198658] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2916.215060] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2916.215389] Lustre: 95282:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2921.448164] Lustre: 95285:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2921.460971] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2921.461499] Lustre: 95285:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 1 previous similar message [ 2921.555465] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2921.889073] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2922.116284] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 2922.229891] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2922.237054] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 2922.575145] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2922.593139] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 2922.603848] Lustre: 101507:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2922.617038] Lustre: 101507:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 7 previous similar messages [ 2923.273670] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2923.288088] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 2923.607367] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2923.615650] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 2924.619674] Lustre: 101507:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2924.641125] Lustre: 101507:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 5 previous similar messages [ 2925.312679] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2925.321961] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 5 previous similar messages [ 2925.656291] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2925.674470] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 5 previous similar messages [ 2928.868990] Lustre: 101507:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2928.880807] Lustre: 101507:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 12 previous similar messages [ 2929.521603] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2929.538886] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 2929.863806] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2929.879324] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 2936.947336] Lustre: 101507:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2936.969128] Lustre: 101507:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 24 previous similar messages [ 2937.619888] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2937.627081] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 24 previous similar messages [ 2937.951168] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2937.967244] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 24 previous similar messages [ 2953.243565] Lustre: 101507:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2953.256264] Lustre: 101507:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 50 previous similar messages [ 2953.890926] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2953.926516] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 2954.255950] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2954.280177] LustreError: 101507:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 2976.287129] Lustre: 101507:0:(ldlm_lib.c:2393:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2976.303764] Lustre: 101507:0:(ldlm_lib.c:2403:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2976.321712] Lustre: 101507:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 74dab0ea-889b-45e9-9175-7a390e4d5c35@192.168.202.55@tcp [ 2976.333471] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2976.338720] Lustre: 101507:0:(ldlm_lib.c:1913:abort_req_replay_queue()) @@@ aborted: req@00000000069ec90b x1873927858096896/t0(8589934767) o101->74dab0ea-889b-45e9-9175-7a390e4d5c35@192.168.202.55@tcp:519/0 lens 592/0 e 0 to 0 dl 1787117984 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2976.377281] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2976.389408] Lustre: lustre-MDT0000: Denying connection for new client 74dab0ea-889b-45e9-9175-7a390e4d5c35 (at 192.168.202.55@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2976.418399] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2976.469811] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2976.476551] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2976.500164] Lustre: Skipped 2 previous similar messages [ 2976.524372] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:76 to 0x280000401:577) [ 2976.528103] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 2992.618726] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2992.618982] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2992.631892] LustreError: Skipped 1 previous similar message [ 2992.633907] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2992.663353] Lustre: Skipped 4 previous similar messages [ 2992.663986] Lustre: Skipped 3 previous similar messages [ 2997.221367] Lustre: server umount lustre-OST0000 complete [ 3007.856414] Lustre: server umount lustre-OST0001 complete [ 3019.533625] Lustre: server umount lustre-MDT0000 complete [ 3023.841584] LustreError: 95282:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3023.862374] LustreError: 95282:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 48 previous similar messages [ 3025.620939] LustreError: 95266:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787118023 with bad export cookie 9006421497065643182 [ 3025.630076] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3025.634221] LustreError: 95266:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3026.325751] Lustre: server umount lustre-MDT0001 complete [ 3040.527646] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 01:40:37 (1787118037) [ 3048.635071] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3048.641273] Lustre: *** cfs_fail_loc=197, val=0*** [ 3064.287767] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3069.407629] LustreError: 103485:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.155@tcp: failed processing log, type 4: rc = -110 [ 3074.665735] Lustre: *** cfs_fail_loc=197, val=0*** [ 3074.669242] Lustre: Skipped 65 previous similar messages [ 3095.327735] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3095.332979] Lustre: Skipped 3 previous similar messages [ 3103.412251] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3112.288951] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3114.766546] Lustre: server umount lustre-OST0000 complete [ 3126.395238] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 01:42:02 (1787118122) [ 3150.377512] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3156.025687] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3166.866393] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3171.949506] LustreError: 105137:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3171.972465] LustreError: 105137:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3171.983786] LustreError: 105137:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 20, retries 0, failed: rc = -5 [ 3172.806029] Lustre: server umount lustre-MDT0000 complete [ 3215.724404] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 3228.146459] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 3267.949428] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3268.186598] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3268.222809] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3268.344851] Lustre: lustre-MDT0000: new disk, initializing [ 3268.458703] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3274.442600] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3287.327118] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3287.528220] Lustre: 109316:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3287.782748] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3292.376637] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3304.590882] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3311.629645] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3321.056685] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3321.622818] Lustre: 111280:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3328.941220] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3330.827650] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3339.394353] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3344.937538] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3345.198705] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3350.567856] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3350.769504] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3356.130759] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3356.141505] LustreError: Skipped 1 previous similar message [ 3356.152356] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3356.186499] Lustre: Skipped 4 previous similar messages [ 3356.208860] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3356.223616] Lustre: Skipped 10 previous similar messages [ 3367.395310] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3367.970693] Lustre: server umount lustre-OST0000 complete [ 3381.068327] Lustre: server umount lustre-MDT0000 complete [ 3382.245427] LustreError: 110476:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3382.281764] LustreError: 110476:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 3387.374953] LustreError: 109309:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787118385 with bad export cookie 9006421497065653738 [ 3387.387596] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3387.398582] LustreError: 109309:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3387.850614] Lustre: server umount lustre-MDT0001 complete [ 3399.739339] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 01:46:36 (1787118396) [ 3410.778558] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 3413.648557] Key type lgssc unregistered [ 3413.960681] LNet: 113771:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3413.968228] LNetError: 113771:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3415.017674] LNet: Removed LNI 192.168.202.155@tcp [ 3415.931830] Key type .llcrypt unregistered [ 3415.934864] Key type ._llcrypt unregistered [ 3433.450332] Key type ._llcrypt registered [ 3433.452890] Key type .llcrypt registered [ 3433.558055] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 3451.522484] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 3452.938348] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3452.981042] alg: No test for adler32 (adler32-zlib) [ 3454.137551] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [ 3454.475370] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 3456.127609] Key type lgssc registered [ 3457.620875] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3505.150673] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3505.213750] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3506.650291] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3506.714795] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3506.812599] Lustre: lustre-MDT0000: new disk, initializing [ 3506.894357] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3507.007540] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3511.889593] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3523.822248] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3524.088701] Lustre: lustre-OST0000: new disk, initializing [ 3524.093815] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3524.098408] Lustre: Skipped 1 previous similar message [ 3524.113494] Lustre: 117948:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3524.184921] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3530.939726] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3531.368817] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3531.377534] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3531.411525] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3542.271713] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3542.458344] Lustre: lustre-OST0001: new disk, initializing [ 3542.461991] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3542.468597] Lustre: 118811:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3542.579830] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3548.855514] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3550.798788] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3550.806967] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3550.851722] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3560.935842] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3560.946407] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3560.960681] Lustre: Skipped 1 previous similar message [ 3560.989602] Lustre: Skipped 1 previous similar message [ 3563.992504] Lustre: server umount lustre-MDT0000 complete [ 3571.386513] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3576.631786] LustreError: 117136:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787118574 with bad export cookie 13624376270607308822 [ 3576.649678] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3576.749744] Lustre: server umount lustre-OST0000 complete [ 3581.025360] Lustre: server umount lustre-OST0001 complete [ 3590.885768] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 01:49:47 (1787118587) [ 3619.531758] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 01:50:15 (1787118615) [ 3634.082927] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 3664.884760] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 01:51:01 (1787118661) [ 3718.659552] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 3735.399849] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 3777.472709] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3777.841601] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3777.885501] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3777.976453] Lustre: lustre-MDT0000: new disk, initializing [ 3778.058402] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3778.099251] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3783.095413] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3793.669684] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3793.827803] Lustre: 126606:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3793.900821] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3793.906951] Lustre: Skipped 1 previous similar message [ 3794.016165] Lustre: lustre-MDT0001: new disk, initializing [ 3794.097799] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3794.131747] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3794.145341] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3799.765370] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3812.129339] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3820.463886] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3829.832736] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3830.166704] Lustre: lustre-OST0000: new disk, initializing [ 3830.173305] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3830.181270] Lustre: 128578:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3830.323272] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3837.981482] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3840.562360] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3840.578473] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3840.691113] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3848.835917] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3853.744760] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3853.952868] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3858.476277] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3858.730224] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3861.473487] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3861.479963] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3861.486266] LustreError: Skipped 1 previous similar message [ 3861.487282] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3861.521822] Lustre: Skipped 1 previous similar message [ 3866.593822] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3866.609702] Lustre: Skipped 1 previous similar message [ 3866.926485] Lustre: server umount lustre-OST0000 complete [ 3871.200600] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3871.207578] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3871.221733] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3871.227366] Lustre: Skipped 1 previous similar message [ 3876.608434] Lustre: server umount lustre-MDT0000 complete [ 3880.545126] LustreError: 128577:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787118878 with bad export cookie 13624376270607310047 [ 3880.550226] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3880.561950] LustreError: 128577:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3881.058118] Lustre: server umount lustre-MDT0001 complete [ 3890.985873] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 01:54:48 (1787118888) [ 3897.435422] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3897.731917] LustreError: 130989:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3897.746266] LustreError: 130989:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3897.817125] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3902.044633] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3902.963929] LustreError: 130990:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3907.041640] LustreError: 130989:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3910.701863] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3915.331308] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3922.284213] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3928.454237] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3935.351782] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3935.648158] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3935.655672] Lustre: Skipped 1 previous similar message [ 3941.775673] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 3950.339926] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3954.683848] Lustre: *** cfs_fail_loc=159, val=0*** [ 3954.688276] Lustre: Skipped 1 previous similar message [ 3971.023314] Lustre: lustre-MDT0000: Client ed3eff47-876a-4714-8d1e-93c8626b94b8 (at 192.168.202.55@tcp) reconnecting [ 3972.641268] Lustre: *** cfs_fail_loc=159, val=0*** [ 3972.652153] Lustre: Skipped 5 previous similar messages [ 3988.924399] Lustre: lustre-MDT0000: Client ed3eff47-876a-4714-8d1e-93c8626b94b8 (at 192.168.202.55@tcp) reconnecting [ 3994.088147] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3994.088686] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3994.100681] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3994.113242] Lustre: Skipped 2 previous similar messages [ 3994.122372] Lustre: Skipped 1 previous similar message [ 3995.921561] Lustre: server umount lustre-OST0000 complete [ 4003.308387] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4003.327213] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4003.348502] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4003.356421] Lustre: Skipped 1 previous similar message [ 4005.886512] Lustre: server umount lustre-MDT0000 complete [ 4009.097391] LustreError: 130971:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787119007 with bad export cookie 13624376270607310922 [ 4009.110648] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4009.619719] Lustre: server umount lustre-MDT0001 complete [ 4020.947074] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 4023.379509] Key type lgssc unregistered [ 4023.646441] LNet: 134764:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4023.661443] LNetError: 134764:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4023.682304] LNet: Removed LNI 192.168.202.155@tcp [ 4024.469211] Key type .llcrypt unregistered [ 4024.472141] Key type ._llcrypt unregistered [ 4047.219403] Key type ._llcrypt registered [ 4047.223323] Key type .llcrypt registered [ 4047.344579] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 4048.106963] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4048.183121] alg: No test for adler32 (adler32-zlib) [ 4049.164024] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [ 4049.350938] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 4050.991275] Key type lgssc registered [ 4051.907785] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4058.519427] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4058.532939] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4059.933918] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4063.283768] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4069.356107] LustreError: 136187:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4069.389289] LustreError: 136187:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4072.354791] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4072.843869] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4077.411826] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4084.228663] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4089.731052] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4095.467847] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4095.739645] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4101.883483] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4104.951823] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4110.178546] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4120.396475] Lustre: *** cfs_fail_loc=159, val=0*** [ 4136.877901] Lustre: lustre-MDT0000: Client 5d501631-4557-4dad-b712-97453aca7c73 (at 192.168.202.55@tcp) reconnecting [ 4138.715590] Lustre: *** cfs_fail_loc=159, val=0*** [ 4155.318171] Lustre: lustre-MDT0001: Client 5d501631-4557-4dad-b712-97453aca7c73 (at 192.168.202.55@tcp) reconnecting [ 4157.242235] Lustre: *** cfs_fail_loc=159, val=0*** [ 4157.247655] Lustre: Skipped 3 previous similar messages [ 4172.719163] Lustre: lustre-MDT0001: Client 5d501631-4557-4dad-b712-97453aca7c73 (at 192.168.202.55@tcp) reconnecting [ 4189.328375] Lustre: 139250:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4192.377977] Lustre: *** cfs_fail_loc=159, val=0*** [ 4192.385814] Lustre: Skipped 4 previous similar messages [ 4207.556864] Lustre: lustre-MDT0000: Client 468942e7-36f0-43d1-a7d1-bb1605690db7 (at 192.168.202.55@tcp) reconnecting [ 4209.411574] Lustre: *** cfs_fail_loc=159, val=0*** [ 4209.418091] Lustre: Skipped 14 previous similar messages [ 4225.982556] Lustre: lustre-MDT0000: Client 468942e7-36f0-43d1-a7d1-bb1605690db7 (at 192.168.202.55@tcp) reconnecting [ 4228.081074] Lustre: 139546:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4248.557889] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4248.576473] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4248.581329] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4248.594825] Lustre: Skipped 1 previous similar message [ 4248.618153] Lustre: Skipped 1 previous similar message [ 4252.952938] Lustre: server umount lustre-OST0000 complete [ 4258.791750] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4258.803104] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4258.803846] Lustre: Skipped 1 previous similar message [ 4263.900204] Lustre: server umount lustre-MDT0000 complete [ 4263.904698] LustreError: 136915:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4263.948919] LustreError: 136915:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4267.971438] LustreError: 136169:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787119265 with bad export cookie 10788713313022076210 [ 4267.974397] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4267.984468] LustreError: 136169:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4268.253954] Lustre: server umount lustre-MDT0001 complete [ 4279.296716] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 4282.045544] Key type lgssc unregistered [ 4282.278607] LNet: 140636:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4282.282554] LNetError: 140636:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4282.305726] LNet: Removed LNI 192.168.202.155@tcp [ 4283.107844] Key type .llcrypt unregistered [ 4283.110780] Key type ._llcrypt unregistered [ 4310.849117] Key type ._llcrypt registered [ 4310.852373] Key type .llcrypt registered [ 4310.928296] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 4311.954657] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4312.111669] alg: No test for adler32 (adler32-zlib) [ 4313.166322] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [ 4313.474914] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 4315.143174] Key type lgssc registered [ 4316.375598] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4324.527356] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4324.545445] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4325.982996] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4330.893869] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4335.589224] LustreError: 142049:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4335.611777] LustreError: 142049:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4340.089498] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4340.443628] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4345.835143] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4355.733645] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4362.589225] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4371.999812] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4372.330295] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4379.590085] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4385.655727] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4387.340653] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4395.889140] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4405.438487] Lustre: 144651:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4409.382264] Lustre: 144699:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4411.150058] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4412.164503] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4416.458670] Lustre: 144845:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4417.179027] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4418.007316] Lustre: 144943:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4426.729603] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4426.745250] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4426.747946] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4426.780173] Lustre: Skipped 1 previous similar message [ 4428.287856] Lustre: server umount lustre-OST0000 complete [ 4432.872486] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4432.882576] LustreError: Skipped 1 previous similar message [ 4432.884605] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4432.905772] Lustre: Skipped 1 previous similar message [ 4432.911248] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4436.960057] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4436.960301] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4438.663538] Lustre: server umount lustre-MDT0000 complete [ 4442.083082] LustreError: 142050:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4442.114913] LustreError: 142050:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4442.609506] LustreError: 142030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787119440 with bad export cookie 11953139842645179357 [ 4442.622942] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4442.624668] LustreError: 142030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4442.985226] Lustre: server umount lustre-MDT0001 complete [ 4453.506976] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 4456.558204] Key type lgssc unregistered [ 4456.851915] LNet: 146004:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4456.869414] LNetError: 146004:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4456.894285] LNet: Removed LNI 192.168.202.155@tcp [ 4457.792638] Key type .llcrypt unregistered [ 4457.794841] Key type ._llcrypt unregistered [ 4480.892955] Key type ._llcrypt registered [ 4480.897420] Key type .llcrypt registered [ 4480.992649] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 4482.025854] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4482.166408] alg: No test for adler32 (adler32-zlib) [ 4483.266741] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [ 4483.492687] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 4485.119774] Key type lgssc registered [ 4486.646260] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4495.490138] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4495.511862] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4497.035421] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4501.736565] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4502.499484] LustreError: 147436:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4502.525320] LustreError: 147436:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 4506.595611] LustreError: 147437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4509.831053] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4510.254275] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4515.257795] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4522.823355] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4531.804104] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4540.143205] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4540.533518] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4541.559114] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4547.473725] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4555.148812] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4555.897085] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4560.381482] Lustre: *** cfs_fail_loc=159, val=0*** [ 4560.385647] Lustre: Skipped 1 previous similar message [ 4576.680409] Lustre: lustre-MDT0000: Client 462ae021-6f3d-4dd2-a5c7-73bcec6f5dbd (at 192.168.202.55@tcp) reconnecting [ 4581.867599] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4581.881190] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4581.891631] Lustre: Skipped 1 previous similar message [ 4581.911602] Lustre: Skipped 1 previous similar message [ 4583.921698] Lustre: server umount lustre-OST0000 complete [ 4592.097827] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4592.101895] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4592.102948] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4592.110806] Lustre: Skipped 1 previous similar message [ 4592.137163] Lustre: Skipped 1 previous similar message [ 4594.232637] Lustre: server umount lustre-MDT0000 complete [ 4597.218714] LustreError: 147436:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4597.247107] LustreError: 147436:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4598.357522] LustreError: 147416:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787119596 with bad export cookie 9208691843841976192 [ 4598.366743] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4598.866679] Lustre: server umount lustre-MDT0001 complete [ 4610.319645] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 4613.272798] Key type lgssc unregistered [ 4613.598868] LNet: 151098:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4613.613040] LNetError: 151098:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4613.635587] LNet: Removed LNI 192.168.202.155@tcp [ 4614.822465] Key type .llcrypt unregistered [ 4614.825799] Key type ._llcrypt unregistered [ 4639.619348] Key type ._llcrypt registered [ 4639.621814] Key type .llcrypt registered [ 4639.710787] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 4640.699569] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4640.854031] alg: No test for adler32 (adler32-zlib) [ 4642.246562] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [ 4642.534706] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 4644.255168] Key type lgssc registered [ 4645.990983] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4655.760562] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4655.773680] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4657.195843] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4662.714481] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4667.371337] LustreError: 152514:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4667.409817] LustreError: 152514:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4671.786340] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4672.320925] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4676.702795] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4684.061702] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4690.279718] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4697.827620] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4698.559368] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4702.663094] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4705.512477] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4713.329438] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4718.014416] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4738.794788] Lustre: lustre-MDT0001: evicting cfdbb0c1-be57-454b-8a65-7534e107d3ae (at 192.168.202.55@tcp) by administrative request [ 4738.800402] Lustre: Skipped 1 previous similar message [ 4769.253113] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4769.254844] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4769.271099] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4769.295604] Lustre: Skipped 1 previous similar message [ 4769.305989] Lustre: Skipped 1 previous similar message [ 4773.555805] Lustre: server umount lustre-OST0000 complete [ 4779.494455] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4779.523286] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4779.530564] Lustre: Skipped 1 previous similar message [ 4783.742325] Lustre: server umount lustre-MDT0000 complete [ 4784.617472] LustreError: 153760:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4784.633407] LustreError: 153760:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4788.490720] LustreError: 152495:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787119786 with bad export cookie 4952595390985408610 [ 4788.503630] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4788.506456] LustreError: 152495:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4789.078562] Lustre: server umount lustre-MDT0001 complete [ 4802.769154] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 4806.175650] Key type lgssc unregistered [ 4806.459241] LNet: 156591:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4806.467521] LNetError: 156591:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4806.492159] LNet: Removed LNI 192.168.202.155@tcp [ 4807.230323] Key type .llcrypt unregistered [ 4807.232961] Key type ._llcrypt unregistered [ 4886.494535] Key type ._llcrypt registered [ 4886.496493] Key type .llcrypt registered [ 4886.587942] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 4887.487934] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4887.638856] alg: No test for adler32 (adler32-zlib) [ 4888.753111] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [ 4888.993993] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 4890.671539] Key type lgssc registered [ 4891.989430] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4900.765920] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4900.804421] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4902.398591] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4907.210839] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4912.612527] LustreError: 159078:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4912.624677] LustreError: 159078:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4915.593943] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4916.009204] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4920.612737] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4928.489664] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4934.579188] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4940.944728] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4941.460054] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4947.640581] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4947.965391] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 4955.410367] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4960.959881] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4971.490428] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4971.498766] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4971.500758] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4971.529251] Lustre: Skipped 1 previous similar message [ 4976.610302] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4976.625738] Lustre: Skipped 1 previous similar message [ 4981.731387] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4981.743195] Lustre: Skipped 1 previous similar message [ 4983.263138] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4983.403626] Lustre: server umount lustre-OST0000 complete [ 4987.877707] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4987.891616] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4987.926094] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4987.932777] Lustre: Skipped 1 previous similar message [ 4991.968227] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4991.970541] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4992.000736] Lustre: Skipped 1 previous similar message [ 4993.314716] Lustre: server umount lustre-MDT0000 complete [ 4997.101535] LustreError: 159810:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4997.131673] LustreError: 159810:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4997.442079] LustreError: 159058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787119995 with bad export cookie 1985338123633015876 [ 4997.458667] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4997.466070] LustreError: 159058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4997.967181] Lustre: server umount lustre-MDT0001 complete [ 5013.700477] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 5017.370927] Key type lgssc unregistered [ 5017.799507] LNet: 162644:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5017.810815] LNetError: 162644:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5018.862276] LNet: Removed LNI 192.168.202.155@tcp [ 5019.755282] Key type .llcrypt unregistered [ 5019.757236] Key type ._llcrypt unregistered [ 5045.995857] Key type ._llcrypt registered [ 5045.998565] Key type .llcrypt registered [ 5046.137991] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 5047.290376] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5047.326853] alg: No test for adler32 (adler32-zlib) [ 5048.472797] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [ 5048.689352] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 5050.337155] Key type lgssc registered [ 5051.529397] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5071.152569] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 02:14:27 (1787120067) [ 5091.432579] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 5103.123228] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 5146.284617] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5146.312981] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5147.670799] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5147.692335] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5147.770334] Lustre: lustre-MDT0000: new disk, initializing [ 5147.851325] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5147.880732] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5153.846990] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 5166.260603] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5166.423164] Lustre: 166912:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5166.615804] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5166.626962] Lustre: Skipped 1 previous similar message [ 5166.853899] Lustre: lustre-MDT0001: new disk, initializing [ 5167.098553] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5167.184575] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5167.211369] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5173.597394] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 5185.638967] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5192.853155] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5199.082381] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5199.348516] Lustre: lustre-OST0000: new disk, initializing [ 5199.351149] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5199.354746] Lustre: 168880:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5199.409376] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5205.061225] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 5208.652430] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5208.662343] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5208.796468] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5213.976975] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5224.418877] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5224.421165] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5224.426920] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5224.432268] LustreError: Skipped 1 previous similar message [ 5224.466784] Lustre: Skipped 1 previous similar message [ 5228.775561] Lustre: server umount lustre-OST0000 complete [ 5233.635784] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5233.649507] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5233.673342] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5233.676063] Lustre: Skipped 1 previous similar message [ 5234.658247] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5234.675218] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5234.685191] Lustre: Skipped 1 previous similar message [ 5239.453849] Lustre: server umount lustre-MDT0000 complete [ 5244.616678] LustreError: 168879:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787120242 with bad export cookie 8810521721851278686 [ 5244.623747] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5244.631903] LustreError: 168879:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5245.236301] Lustre: server umount lustre-MDT0001 complete [ 5256.716971] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 5259.543425] Key type lgssc unregistered [ 5259.805756] LNet: 170785:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5259.821074] LNetError: 170785:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5259.850063] LNet: Removed LNI 192.168.202.155@tcp [ 5261.058274] Key type .llcrypt unregistered [ 5261.063684] Key type ._llcrypt unregistered [ 5283.575914] Key type ._llcrypt registered [ 5283.578915] Key type .llcrypt registered [ 5283.751706] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 5284.548505] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5284.575947] alg: No test for adler32 (adler32-zlib) [ 5285.649490] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [ 5285.952304] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 5287.703229] Key type lgssc registered [ 5288.919117] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5296.295044] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5296.333531] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5297.988803] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5301.895679] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 5307.399315] LustreError: 172199:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5307.422400] LustreError: 172199:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5311.158963] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5311.500452] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5316.870825] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 5325.534918] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5331.679225] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5339.464756] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5339.929391] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5343.043388] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5345.656163] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 5353.260163] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5363.535906] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5363.549404] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5363.558932] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5363.563670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.572412] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5364.302334] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5364.319689] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5364.327685] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5364.332129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.341646] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5365.141857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.853229] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5365.856927] Lustre: Skipped 1 previous similar message [ 5365.869390] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5365.880841] Lustre: Skipped 1 previous similar message [ 5365.885085] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5365.888759] Lustre: Skipped 1 previous similar message [ 5365.892715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.905106] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5365.911100] Lustre: Skipped 1 previous similar message [ 5366.706162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.514725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.141163] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5368.146553] Lustre: Skipped 2 previous similar messages [ 5368.160836] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5368.165725] Lustre: Skipped 2 previous similar messages [ 5368.168801] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5368.173811] Lustre: Skipped 2 previous similar messages [ 5368.178996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.185394] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5368.187812] Lustre: Skipped 2 previous similar messages [ 5368.906126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.676919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.455562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.270643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.021926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.720269] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5372.724456] Lustre: Skipped 5 previous similar messages [ 5372.734701] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5372.740691] Lustre: Skipped 5 previous similar messages [ 5372.742767] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5372.745621] Lustre: Skipped 5 previous similar messages [ 5372.748569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.754752] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5372.760168] Lustre: Skipped 5 previous similar messages [ 5373.471435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.322031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.199668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.094187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.891824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.637347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.324440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.014330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.807693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.720598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.553679] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5381.559424] Lustre: Skipped 10 previous similar messages [ 5381.580797] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5381.599536] Lustre: Skipped 10 previous similar messages [ 5381.606519] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5381.615963] Lustre: Skipped 10 previous similar messages [ 5381.618922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.645838] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5381.651882] Lustre: Skipped 10 previous similar messages [ 5382.481483] LustreError: 172219:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5382.496943] LustreError: 172219:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5382.497167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.424974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.257950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.209131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.119442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.809645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.625172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.504145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.479443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.233983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.202907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.072763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.923574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.689747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.557527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.309422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.961824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.659821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.445140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.226690] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5398.230581] Lustre: Skipped 19 previous similar messages [ 5398.245546] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5398.253683] Lustre: Skipped 19 previous similar messages [ 5398.256298] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5398.261364] Lustre: Skipped 19 previous similar messages [ 5398.265837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.276868] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5398.282438] Lustre: Skipped 19 previous similar messages [ 5399.109573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.724946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.283806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.874616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.657293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.334700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.204620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.825726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.770315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.587924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.288529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.009809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.933445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.728423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.626454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.385097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.103702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.851350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.558516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.320258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.161244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.955954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.703646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.412481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.225432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.841529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.540400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.213124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.029262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.861400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.613520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.318222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.109375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.802824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.475234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.382984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.259651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.013346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.781224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.644397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.530587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.283188] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5430.288845] Lustre: Skipped 41 previous similar messages [ 5430.303835] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5430.315736] Lustre: Skipped 41 previous similar messages [ 5430.319299] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5430.324967] Lustre: Skipped 41 previous similar messages [ 5430.328841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.339712] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5430.345494] Lustre: Skipped 41 previous similar messages [ 5431.174568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.108627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.829916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.561964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.360277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.170942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.871128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.648541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.451320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.358379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.148776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.951643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.730386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.499660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.357807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.217477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.039152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.007636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.971524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.824513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.544314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.532431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.475752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.377576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.160799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.892750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.716907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.563764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.268625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.142689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.795878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.565531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.278406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.152973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.062211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.841856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.580892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.335776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.383044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.516843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.506098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.576457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.652549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.342279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.223088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.050416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.019693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.857712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.764821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.840376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.775241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.551811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.189645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.986785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.804113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.592498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.394967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.251879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.077875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.958266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.883327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.788574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.871173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.804309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.782327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.593810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.454025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.242721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.046241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.866810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.726992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.460708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.253684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.073653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.911579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.634986] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5494.642619] Lustre: Skipped 75 previous similar messages [ 5494.667994] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5494.684781] Lustre: Skipped 75 previous similar messages [ 5494.692407] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5494.702031] Lustre: Skipped 75 previous similar messages [ 5494.707766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.724531] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5494.734392] Lustre: Skipped 75 previous similar messages [ 5495.555355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.366980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.212926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.122976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.943837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.768674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.700382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.368562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.200520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.942862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.867039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.947033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.969485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.892300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.699860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.484928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.300891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.051170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.996818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.870670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.593287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.444490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.227963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.979762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.733123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.441668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.254320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.096968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.881517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.767533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.784056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.751740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.511985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.494885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.432818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.385549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.334826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.475769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.483658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.400690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.396682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.178725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.949676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.776070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.582598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.420626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.324827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.389669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.281198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.118036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.008558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.876514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.780494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.681260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.635442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.583066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.506593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.239462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.157091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.267857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.272561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.122364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.885102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.873150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.770316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.601698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.491584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.497629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.467562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.594595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.760207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.822965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.644879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.532663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.348680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.391524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.206605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.124468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.975630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.149784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.098667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.100721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.096087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.980365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.104114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.915212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.861851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.875205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.003931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.959726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.760564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.495527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.233975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.337594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.265353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.169278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.956811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.891648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.884781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.647864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.507437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.290794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.017812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.872478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.615892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.700468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.535608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.363827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.319958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.261611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.965665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.895665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.707224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.717172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.688475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.598766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.334473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.243772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.236307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.262681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.096364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.063437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.828540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.723730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.382422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.152546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.980937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.809022] LustreError: 172219:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5608.814618] LustreError: 172219:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5608.833675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.497957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.272523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.951806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.882552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.842557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.831717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.767912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.784620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.686875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.471087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.321674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.307159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.159780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.985335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.793935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.846165] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5622.851055] Lustre: Skipped 143 previous similar messages [ 5622.861665] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5622.867767] Lustre: Skipped 143 previous similar messages [ 5622.872049] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5622.876825] Lustre: Skipped 143 previous similar messages [ 5622.880723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.890290] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5622.893269] Lustre: Skipped 143 previous similar messages [ 5623.717560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.500877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.340356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.239930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.255538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.074464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.924751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.919098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.866745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.789122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.638873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.567748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.433667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.419613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.317299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.231835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.976684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.738301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.513732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.428384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.393419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.395906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.252725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.119425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.056662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.905232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.727832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.931514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.764294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.648431] LustreError: 172219:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5649.656328] LustreError: 172219:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5649.674092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.490814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.413834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.256318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.170100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.119888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.842882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.568356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.269505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.083819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.961990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.738218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.820379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.656432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.582219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.338727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.353355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.450397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.293736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.269773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.297502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.432819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.500812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.492502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.466336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.439982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.407549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.344565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.298809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.259354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.160329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.070612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.857848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.826578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.846596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.881596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.810840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.724974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.694973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.590891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.430458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.192529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.917075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.960276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.979663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.111620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.109572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.102645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.026091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.891841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.013099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.879055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.774632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.609064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.422889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.225875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.139882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.325981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.210082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.025936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.885132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.649388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.441077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.373138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.408128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.278110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.133863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.012926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.017333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.029857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.966223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.732333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.701869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.637384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.516493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.295074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.188827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.974803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.753347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.512098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.294782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.010905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.899817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.670990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.611905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.354574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.129791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.918317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.711418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.630755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.698261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.483385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.471978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.488090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.491235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.362911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.197607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.017804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.806807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.553469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.433524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.242931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.250515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.043605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.289338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.326826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.277782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.100801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.920926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.912037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.704917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.383848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.102591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.868940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.818891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.570760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.402562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.194958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.222692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.133519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.971808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.853179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.670439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.594613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.380388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.222883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.089799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.051729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.985542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.091551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.976201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.010764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.099713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.799796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.806171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.594708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.713485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.565453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.446608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.215410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.027452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.808544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.701446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.442885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.178950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.090672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.953174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.892439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.984694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.019029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.155557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.168113] LustreError: 172219:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5785.181441] LustreError: 172219:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5785.186102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.869873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.701564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.624749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.660405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.520545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.466884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.673287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.500292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.565790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.521093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.410777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.293065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.273182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.186654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.120386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.881870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.667498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.354796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.230250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.948630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.654936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.404053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.232148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.094616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.994190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.867319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.664960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.513606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.276859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.001271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.740364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.575653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.527799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.494377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.699551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.649940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.822856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.035873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.152654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.360731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.198604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.080885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.842658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.664795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.512976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.378406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.107240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.148462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.993783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.894951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.709750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.475859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.378292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.401640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.405054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.245654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.176449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.059531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.002483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.929655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.870384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.849652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.804832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.695819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.740434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.475902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.435670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.383206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.369505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.272501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.012873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.940768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.863969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.681561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.433877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.093467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.883387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.657598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.481141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.212890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.161106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.858643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.696568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.428788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.254099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.942615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.734689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.445276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.281667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.180766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.087727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.983089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.865353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.685509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.728096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.517435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.301322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.050158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.948492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.824739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.788806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.626926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.561797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.456317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.187880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.841104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.672752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.445001] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5879.452317] Lustre: Skipped 288 previous similar messages [ 5879.467951] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5879.477510] Lustre: Skipped 288 previous similar messages [ 5879.480745] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5879.484425] Lustre: Skipped 288 previous similar messages [ 5879.491265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.500666] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5879.505723] Lustre: Skipped 288 previous similar messages [ 5880.421517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.080682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.799848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.690180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.641468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.534368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.727459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.657304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.483484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.140949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.891503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.732195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.487503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.263888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.067899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.046543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.135915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.066282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.033757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.930852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.868453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.787611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.673255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.585393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.429807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.298369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.102453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.897557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.714971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.414373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.151331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.120687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.114356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.115310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.986809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.025287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.029327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.930702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.731616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.395281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.287238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.000870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.785946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.629155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.430182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.360321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.385499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.536711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.624435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.540267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.518875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.449354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.471792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.422141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.400669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.352992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.441516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.353577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.369708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.372063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.717231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.042758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.130863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.115642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.323433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.428849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.357113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.329928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.384064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.548964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.943153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.976696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.977416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.964869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.837855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.942435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.025170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.233561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.158494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.128648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.119716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.379245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.514858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.565781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.429264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.521678] LustreError: 172219:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5961.533895] LustreError: 172219:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5961.537285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.419350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.491688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.412484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.491489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.420572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.328152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.173194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.055702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.788070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.864529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.666718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.628470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.647962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.046749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.033628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.026835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.057594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.037863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.801928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.801805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.795274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.900478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.942059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.209495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.553465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.501552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.760653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.894318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.859414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.954490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.736070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.666743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.616572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.487516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.471392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.356747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.336756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.262239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.298480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.231502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.301851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.462499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.458049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.200561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.064342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.886285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.781193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.782821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.627792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.652194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.533983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.434642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.359788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.247327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.040741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.075172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.939637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.883231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.545380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.322611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.233830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.147122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.921256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.035640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.768063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.761534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.673924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.662023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.517158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.437885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.458598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.248325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.215097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.491993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.421950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.299679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.127995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.194272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.012597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.287761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.746599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.987409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.807502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.237826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.458815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.707707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.679311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.097545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.514422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.812082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.593862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.364971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.298838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.314234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.245284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.698809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.153773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.512519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.815062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.869371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.980663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.920080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.025402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.171903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.089896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.081920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.047715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.873499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.976533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.908499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.761085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.614483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.480315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.264552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.227107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.083402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.966568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.838762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.767127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.815504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.743337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.760552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.824177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.762539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.795474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.606743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.776248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.796340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.908432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.935252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.133996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.124380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.351616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.565221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.586821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.600819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.689951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.699687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.561782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.507463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.300483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.414889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.309926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.445148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.385596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.614312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.520301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.904222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.579354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.063401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.238499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.478740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.556634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.702683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.151780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.207259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.556332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.550335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.814123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.975357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.103848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.269733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.223459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.240329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.121858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.147236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.281199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.101984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.050086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.968658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.010911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.025662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.011466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.903368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.965386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.952240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.136763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.172854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.273918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.334651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.343801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.359486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.345850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.315526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.291214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.135330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.186811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.220584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.108287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.907288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.728262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.562901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.348876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.242476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.009423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.979655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.913571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.763687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.812241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.770801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.813101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.775601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.985472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.110679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.209119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.083821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.142553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.162209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.388331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.604463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.672382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.510460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.588191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.654593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.756588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.956267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.170273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.299295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.220219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.295842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.380507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.420657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.729528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.555322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.098335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.671630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.884341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.029605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.076664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.147696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.472529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.705408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.498533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.728430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.670495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.839042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.160043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.079887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.288960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.441993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.443934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.543766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.273731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.514898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.467935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.262732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.187572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.041274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.883697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.695522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.505136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.525085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.533150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.910145] LustreError: 172219:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6221.921673] LustreError: 172219:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6221.936786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.010945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.093684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.150203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.218856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.329363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.641933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.609697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.657676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.883392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.725236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.840527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.819774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.878107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.960100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.919518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.989543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.927152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.325952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.598769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.672294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.607374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.043450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.242443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.576687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.778708] LustreError: 172219:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6249.800929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.803021] LustreError: 172219:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6251.022223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.955399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.244311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.439404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.642785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.704764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.097296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.226940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.421777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.457979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.411781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.191932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.035568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.851780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.807592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.901447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.773780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.684808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.767035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.786155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.826037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.772732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.575615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.047509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.244190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.189739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.254592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.157203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.478535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.618402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.650330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.114934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.103887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.246598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.996915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.043988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.078252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.070155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.191234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.286667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6717.932679] LustreError: 174023:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6717.954820] LustreError: 172219:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7424.994410] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7425.000877] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7425.009415] LustreError: Skipped 1 previous similar message [ 7425.021154] Lustre: Skipped 405 previous similar messages [ 7425.028300] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7426.016564] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7426.028155] LustreError: Skipped 3 previous similar messages [ 7426.036238] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7426.365544] Lustre: server umount lustre-OST0000 complete [ 7430.793686] Lustre: server umount lustre-MDT0000 complete [ 7431.136264] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7431.144161] LustreError: 172199:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7433.200659] LustreError: 172927:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7433.219243] LustreError: 172927:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7434.808138] LustreError: 172180:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787122432 with bad export cookie 17363959739081200802 [ 7434.809641] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7434.819131] LustreError: 172180:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7437.399326] Lustre: server umount lustre-MDT0001 complete [ 7449.233888] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 7451.799612] Key type lgssc unregistered [ 7452.077501] LNet: 227460:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7452.091731] LNetError: 227460:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7452.110526] LNet: Removed LNI 192.168.202.155@tcp [ 7453.077910] Key type .llcrypt unregistered [ 7453.081824] Key type ._llcrypt unregistered [ 7478.556742] Key type ._llcrypt registered [ 7478.560831] Key type .llcrypt registered [ 7478.679878] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 7480.241960] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7480.267815] alg: No test for adler32 (adler32-zlib) [ 7481.432275] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [ 7481.715267] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 7483.423214] Key type lgssc registered [ 7485.006521] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7495.230371] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7495.272320] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7497.048622] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7502.630251] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7506.413230] LustreError: 228878:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7506.450465] LustreError: 228878:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7511.523128] LustreError: 228879:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7512.167197] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7512.546830] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7517.581290] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7526.360686] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7533.644043] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7543.271960] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7543.585805] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7549.588696] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7558.311566] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7566.138575] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7574.867731] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7577.140734] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7577.158295] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7577.182491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7577.195241] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7577.198489] LustreError: 228898:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7580.565743] Lustre: lustre-OST0000: Not available for connect from 192.168.202.55@tcp (stopping) [ 7581.677949] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7581.687261] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7581.693589] Lustre: Skipped 1 previous similar message [ 7581.705252] Lustre: Skipped 1 previous similar message [ 7585.650331] Lustre: lustre-OST0000: Not available for connect from 192.168.202.55@tcp (stopping) [ 7586.034727] Lustre: server umount lustre-OST0000 complete [ 7597.288583] LustreError: 228898:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7622.793215] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7623.223568] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7629.853651] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7637.222404] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7638.955220] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7645.180716] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9ba9c6971000.ost_server_uuid 50 [ 7646.659455] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9ba9c6971000.ost_server_uuid in FULL state after 0 sec [ 7646.759553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7646.768447] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7646.769090] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7646.788865] Lustre: Skipped 1 previous similar message [ 7651.812560] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7651.821691] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7651.832622] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7656.940892] Lustre: server umount lustre-OST0000 complete [ 7661.029102] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7661.040065] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7661.050097] Lustre: Skipped 1 previous similar message [ 7661.059532] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7661.074633] Lustre: Skipped 1 previous similar message [ 7666.930217] Lustre: server umount lustre-MDT0000 complete [ 7667.178112] LustreError: 228879:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7667.197555] LustreError: 228879:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7670.996136] LustreError: 228861:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787122668 with bad export cookie 13681933425458318938 [ 7671.003285] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7671.005201] LustreError: 228861:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7671.255610] Lustre: server umount lustre-MDT0001 complete [ 7685.852226] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 7689.485787] Key type lgssc unregistered [ 7689.831277] LNet: 234112:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7689.852551] LNetError: 234112:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7690.917402] LNet: Removed LNI 192.168.202.155@tcp [ 7691.845461] Key type .llcrypt unregistered [ 7691.847420] Key type ._llcrypt unregistered [ 7727.769641] Key type ._llcrypt registered [ 7727.770719] Key type .llcrypt registered [ 7727.856280] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 7728.660359] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7728.867572] alg: No test for adler32 (adler32-zlib) [ 7730.004493] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [ 7730.225303] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 7731.887197] Key type lgssc registered [ 7733.380350] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7744.723529] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7744.760955] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7746.357959] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7751.250613] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7756.769966] LustreError: 235842:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7756.786603] LustreError: 235842:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7760.495554] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7760.897817] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7765.276481] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7768.214189] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7775.341636] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7775.813805] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7782.781121] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7791.145856] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7791.825508] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7792.031588] Lustre: lustre-OST0001: new disk, initializing [ 7792.034109] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7792.039778] Lustre: 238155:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7792.135247] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7799.544286] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7801.026448] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7801.038121] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7801.135969] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7814.097864] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7821.409952] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7842.297842] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7842.300517] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7842.330916] Lustre: Skipped 3 previous similar messages [ 7842.368219] Lustre: Skipped 3 previous similar messages [ 7845.347601] Lustre: server umount lustre-MDT0000 complete [ 7847.406789] LustreError: 239175:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7847.448746] LustreError: 239175:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7850.247635] LustreError: 235823:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787122848 with bad export cookie 16596977035112905448 [ 7850.254226] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7850.260975] LustreError: 235823:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7851.030570] Lustre: server umount lustre-MDT0001 complete [ 7867.465791] Lustre: server umount lustre-OST0000 complete [ 7868.077446] Lustre: 235105:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787122850/real 1787122850] req@ffff94441e311180 x1873934001732992/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787122866 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7868.148141] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7871.456977] Lustre: 235107:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787122853/real 1787122853] req@ffff94441e313800 x1873934001733248/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787122869 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7872.146882] Lustre: server umount lustre-OST0001 complete [ 7895.852329] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 7906.687744] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7906.956818] LustreError: 241275:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7907.014982] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7911.131896] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7912.420292] LustreError: 241276:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7912.446189] LustreError: 241276:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7916.520094] LustreError: 241275:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7919.775639] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7920.250280] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7924.660419] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7927.809860] Lustre: 242416:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7934.968827] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7942.275932] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7945.526304] LustreError: 242772:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7951.880294] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 7952.272474] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7952.574973] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7952.582817] Lustre: Skipped 1 previous similar message [ 7959.380408] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 7967.882280] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7972.641345] Lustre: 244288:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7986.657393] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7986.670978] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7986.717772] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7988.709996] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7988.730491] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7988.745758] Lustre: Skipped 1 previous similar message [ 7988.759858] Lustre: Skipped 3 previous similar messages [ 7991.633066] Lustre: server umount mylustre-MDT0000 complete [ 7993.831790] LustreError: 241276:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7993.860720] LustreError: 241276:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 7995.679426] LustreError: 242418:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787122993 with bad export cookie 16596977035112908255 [ 7995.679732] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7995.692157] LustreError: 242418:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7996.175597] Lustre: server umount mylustre-MDT0001 complete [ 8010.798413] Lustre: server umount mylustre-OST0000 complete [ 8014.303160] Lustre: 235108:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787122996/real 1787122996] req@ffff94441ff75500 x1873934001795840/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787123012 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8014.329202] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8014.340943] Lustre: Skipped 1 previous similar message [ 8014.869987] Lustre: server umount mylustre-OST0001 complete [ 8041.920507] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 8054.138948] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8054.440612] LustreError: 246595:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8054.493910] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8059.627704] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8068.726968] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8074.445365] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8077.768755] Lustre: 247738:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8086.007290] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8094.224689] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8099.781881] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 8105.333303] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8114.379217] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8122.214742] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8125.677670] Lustre: 249610:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8134.752951] Lustre: server umount tfs-MDT0000 complete [ 8136.163486] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8136.169083] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8136.178864] LustreError: 246595:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8136.188854] LustreError: 246595:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 8138.567741] LustreError: 246570:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787123136 with bad export cookie 16596977035112909914 [ 8138.571594] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8138.593972] LustreError: 246570:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8138.859221] Lustre: server umount tfs-MDT0001 complete [ 8153.095735] Lustre: server umount tfs-OST0000 complete [ 8167.191872] Lustre: server umount tfs-OST0001 complete [ 8192.733726] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 8203.751578] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8204.110187] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8204.117319] Lustre: Skipped 3 previous similar messages [ 8208.658159] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8218.202333] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8223.898621] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8227.207919] Lustre: 253003:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8235.003422] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8241.827725] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8249.438451] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8250.757895] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 8256.535717] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8268.152671] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8272.479774] Lustre: 254874:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8280.500599] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 03:07:57 (1787123277) [ 8301.029885] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8301.036291] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8301.046231] Lustre: Skipped 3 previous similar messages [ 8301.054912] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8304.555896] Lustre: server umount lustre-MDT0000 complete [ 8307.168983] LustreError: 252595:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8307.203819] LustreError: 252595:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 11 previous similar messages [ 8308.643035] LustreError: 251837:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787123306 with bad export cookie 16596977035112911363 [ 8308.644761] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8308.656212] LustreError: 251837:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8312.292316] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8312.299734] Lustre: Skipped 4 previous similar messages [ 8315.099990] Lustre: server umount lustre-MDT0001 complete [ 8325.846408] Lustre: server umount lustre-OST0000 complete [ 8345.055274] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8345.471788] Lustre: server umount lustre-OST0001 complete [ 8353.967490] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 8365.210911] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 8405.373222] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 8405.373222] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8405.373222] [ 8405.396768] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8405.685687] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8405.715864] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8405.771525] Lustre: lustre-MDT0000: new disk, initializing [ 8405.834529] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8405.839298] Lustre: Skipped 3 previous similar messages [ 8405.847913] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8410.754393] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8421.877199] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 8421.877199] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8421.877199] [ 8421.939067] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8422.109171] Lustre: 258061:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8422.174570] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8422.181902] Lustre: Skipped 1 previous similar message [ 8422.271553] Lustre: lustre-MDT0001: new disk, initializing [ 8422.375931] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8422.388349] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8426.874741] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8438.986890] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8439.341518] Lustre: lustre-OST0000: new disk, initializing [ 8439.345532] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8439.355576] Lustre: 259643:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8441.413282] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8441.426183] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8441.622821] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8445.438988] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8456.801609] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8457.101621] Lustre: lustre-OST0001: new disk, initializing [ 8457.106725] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8457.115307] Lustre: 260515:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8458.838680] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8458.856436] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8459.057188] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8465.871676] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8479.885887] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 03:11:16 (1787123476) [ 8498.823722] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 03:11:35 (1787123495) [ 8515.560849] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8515.584850] Lustre: Skipped 6 previous similar messages [ 8515.586275] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8515.599805] Lustre: Skipped 4 previous similar messages [ 8520.425145] Lustre: server umount lustre-OST0000 complete [ 8524.768575] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8531.502171] Lustre: server umount lustre-MDT0000 complete [ 8536.289419] LustreError: 260514:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787123534 with bad export cookie 16596977035112912693 [ 8536.294414] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8536.297589] LustreError: 260514:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8541.156548] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8541.164738] Lustre: Skipped 7 previous similar messages [ 8542.832613] Lustre: server umount lustre-MDT0001 complete [ 8557.676902] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 8571.581861] Lustre: server umount lustre-OST0001 complete [ 8580.852792] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 8600.428534] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 8640.539888] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8640.829613] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8640.867393] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8640.952877] Lustre: lustre-MDT0000: new disk, initializing [ 8641.025458] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8645.931976] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8657.543968] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8657.724432] Lustre: 265075:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8657.804049] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8657.807598] Lustre: Skipped 1 previous similar message [ 8657.901251] Lustre: lustre-MDT0001: new disk, initializing [ 8658.043835] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8658.058834] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8662.531819] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8673.155463] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8680.629406] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8688.637579] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8689.037597] Lustre: lustre-OST0000: new disk, initializing [ 8689.042653] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8689.052925] Lustre: 267047:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8689.268150] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8689.281195] Lustre: Skipped 5 previous similar messages [ 8690.609772] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8690.634769] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8690.835214] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8697.446493] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8708.629867] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8716.256667] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8716.260842] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8716.282386] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8716.314515] Lustre: Skipped 5 previous similar messages [ 8721.165761] Lustre: server umount lustre-OST0000 complete [ 8731.616440] LustreError: 265955:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8731.625531] LustreError: 265955:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 13 previous similar messages [ 8731.771580] Lustre: server umount lustre-MDT0000 complete [ 8737.045470] LustreError: 265873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787123735 with bad export cookie 16596977035112914163 [ 8737.049939] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8737.060495] LustreError: 265873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8737.599507] Lustre: server umount lustre-MDT0001 complete [ 8751.755562] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 8754.604487] Key type lgssc unregistered [ 8754.984354] LNet: 268807:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8754.992310] LNetError: 268807:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8756.077159] LNet: Removed LNI 192.168.202.155@tcp [ 8757.206216] Key type .llcrypt unregistered [ 8757.211975] Key type ._llcrypt unregistered [ 8789.421852] Key type ._llcrypt registered [ 8789.426447] Key type .llcrypt registered [ 8789.558508] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 8790.931806] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8790.979990] alg: No test for adler32 (adler32-zlib) [ 8792.102180] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [ 8792.277862] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 8793.943336] Key type lgssc registered [ 8795.192023] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8803.250381] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8803.270648] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8804.746185] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8809.550297] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8814.050925] LustreError: 270287:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8814.075772] LustreError: 270287:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8819.174158] LustreError: 270288:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8820.171152] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8820.515508] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8825.663458] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8835.019294] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8844.948753] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8853.044996] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8853.550858] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8860.715637] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 8869.419354] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8870.380556] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8874.338865] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8874.342244] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8874.383118] Lustre: Skipped 1 previous similar message [ 8878.643613] Lustre: server umount lustre-OST0000 complete [ 8884.705275] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8884.709842] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8884.722381] Lustre: Skipped 1 previous similar message [ 8884.732820] Lustre: Skipped 1 previous similar message [ 8888.594659] Lustre: server umount lustre-MDT0000 complete [ 8889.830444] LustreError: 271016:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8889.849918] LustreError: 271016:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8892.305972] LustreError: 270267:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787123890 with bad export cookie 4788588216337546469 [ 8892.313358] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8892.324090] LustreError: 270267:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8892.659052] Lustre: server umount lustre-MDT0001 complete [ 8904.510287] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 8907.555222] Key type lgssc unregistered [ 8907.881792] LNet: 273874:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8907.894387] LNetError: 273874:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8907.934392] LNet: Removed LNI 192.168.202.155@tcp [ 8908.611878] Key type .llcrypt unregistered [ 8908.614776] Key type ._llcrypt unregistered [ 8925.765037] Key type ._llcrypt registered [ 8925.767631] Key type .llcrypt registered [ 8925.890514] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 8927.328237] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8927.369245] alg: No test for adler32 (adler32-zlib) [ 8928.556493] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [ 8928.764214] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 8930.503209] Key type lgssc registered [ 8931.790884] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8942.433389] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8942.464791] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8958.111532] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8963.231383] LustreError: 275071:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.155@tcp: failed processing log, type 4: rc = -110 [ 8989.919233] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8997.122491] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9007.142504] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9009.421676] Lustre: server umount lustre-OST0000 complete [ 9028.633603] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 9031.272376] Key type lgssc unregistered [ 9031.521625] LNet: 276702:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9031.525972] LNetError: 276702:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9031.532775] LNet: Removed LNI 192.168.202.155@tcp [ 9032.180852] Key type .llcrypt unregistered [ 9032.182941] Key type ._llcrypt unregistered [ 9079.326921] Key type ._llcrypt registered [ 9079.328783] Key type .llcrypt registered [ 9079.439885] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 9081.005124] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9081.121225] alg: No test for adler32 (adler32-zlib) [ 9082.417887] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [ 9082.654392] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 9084.351189] Key type lgssc registered [ 9085.584967] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9099.362542] loop0: detected capacity change from 0 to 204800000 [ 9099.552321] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9099.561226] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9099.948791] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9101.034676] loop0: detected capacity change from 0 to 204800000 [ 9101.283941] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9101.295483] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9101.777564] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9102.853861] loop0: detected capacity change from 0 to 204800000 [ 9103.001263] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9103.014813] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9103.143166] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9104.337978] loop0: detected capacity change from 0 to 204800000 [ 9104.544420] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9104.574643] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9104.721530] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9105.751756] loop0: detected capacity change from 0 to 204800000 [ 9105.781568] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9109.818183] loop0: detected capacity change from 0 to 204800000 [ 9109.839695] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9113.503728] loop0: detected capacity change from 0 to 204800000 [ 9113.547756] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9117.794858] loop0: detected capacity change from 0 to 204800000 [ 9117.849368] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9122.320226] loop0: detected capacity change from 0 to 204800000 [ 9122.415919] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9122.485634] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9122.562787] Lustre: 280123:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9122.569987] Lustre: 280123:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9122.584733] Lustre: 280123:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9122.597682] Lustre: 280123:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9122.606840] Lustre: 280123:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9122.617898] Lustre: 280123:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9122.629611] Lustre: 280123:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9122.641583] Lustre: 280123:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9122.653356] Lustre: 280123:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9122.663845] Lustre: 280123:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9122.669496] Lustre: 280123:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9122.676531] Lustre: 280123:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9122.687368] Lustre: 280123:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9122.695241] Lustre: 280123:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9129.069413] Lustre: server umount MGS complete [ 9129.965511] loop0: detected capacity change from 0 to 204800000 [ 9130.154857] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9130.644472] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9130.682893] Lustre: lustre-MDT0000: Aborting client recovery [ 9131.571724] loop1: detected capacity change from 0 to 204800000 [ 9131.767531] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9131.909175] Lustre: 280554:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9131.919867] Lustre: 280554:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9131.935107] Lustre: 280554:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9131.948875] Lustre: 280554:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9131.973594] Lustre: 280554:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9131.986665] Lustre: 280554:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9132.005771] Lustre: 280554:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9132.028044] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9132.279911] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9132.382166] Lustre: lustre-MDT0001: Aborting client recovery [ 9133.641109] loop2: detected capacity change from 0 to 204800000 [ 9133.905685] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9134.022322] Lustre: 280633:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9134.044804] Lustre: 280633:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9134.055097] Lustre: 280633:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9134.062426] Lustre: 280633:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9134.070694] Lustre: 280633:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9134.080521] Lustre: 280633:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9134.090641] Lustre: 280633:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9134.169264] Lustre: Found index 0 for lustre-OST0000, updating log [ 9134.194413] Lustre: 280652:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9134.263906] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9134.282625] Lustre: lustre-OST0000: Aborting recovery [ 9135.252422] loop3: detected capacity change from 0 to 204800000 [ 9135.419263] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9135.545410] Lustre: 280718:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9135.580482] Lustre: 280718:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9135.608361] Lustre: 280718:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9135.646080] Lustre: 280718:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9135.662264] Lustre: 280718:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9135.680627] Lustre: 280718:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9135.702735] Lustre: 280718:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9135.748579] Lustre: Found index 1 for lustre-OST0001, updating log [ 9135.801451] Lustre: 280724:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9135.892221] Lustre: lustre-OST0001: Aborting recovery [ 9141.034074] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9141.035136] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9141.069023] LustreError: 280498:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 11, retries 0, failed: rc = -108 [ 9141.073811] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9141.085444] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9142.067393] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9142.071817] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9145.565770] LustreError: 280465:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 540us: rc = -2 [ 9146.075182] LustreError: 280463:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 2484us: rc = -2 [ 9146.098816] LustreError: 280463:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 18 previous similar messages [ 9162.731708] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9162.740280] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9162.753072] Lustre: Skipped 1 previous similar message [ 9162.765056] Lustre: Skipped 1 previous similar message [ 9165.395815] Lustre: server umount lustre-MDT0000 complete [ 9166.591791] LustreError: 280448:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787124164 with bad export cookie 12430550781094692132 [ 9166.597051] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9166.607038] LustreError: 280448:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9167.849064] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9167.850596] LustreError: 280462:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9167.855298] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9167.863784] Lustre: Skipped 3 previous similar messages [ 9167.922482] LustreError: 280462:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9171.876808] LustreError: 280956:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9171.883086] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9171.920245] LustreError: 280956:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9171.946350] Lustre: Skipped 2 previous similar messages [ 9173.220975] Lustre: server umount lustre-MDT0001 complete [ 9180.627229] Lustre: server umount lustre-OST0000 complete [ 9187.786466] Lustre: server umount lustre-OST0001 complete [ 9196.823408] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 03:23:13 (1787124193) [ 9232.646725] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 9245.496293] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 9289.765630] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9289.984701] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9290.020919] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9290.128804] Lustre: lustre-MDT0000: new disk, initializing [ 9290.249145] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9290.258406] Lustre: Skipped 1 previous similar message [ 9290.299272] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9295.856125] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9308.215029] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9308.321355] Lustre: 284796:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9308.358440] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9308.361478] Lustre: Skipped 1 previous similar message [ 9308.419769] Lustre: lustre-MDT0001: new disk, initializing [ 9308.515963] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9308.610808] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9308.629939] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9313.393640] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9323.895770] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9330.971906] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9338.819677] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9339.258386] Lustre: lustre-OST0000: new disk, initializing [ 9339.263484] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9339.270544] Lustre: 286764:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9339.372314] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9345.958047] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9346.640384] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9346.658582] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9346.783667] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9355.872894] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9362.023462] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9371.878675] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9377.094489] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9418.737966] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9418.749202] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9418.754021] LustreError: Skipped 1 previous similar message [ 9418.756136] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9418.772306] Lustre: Skipped 1 previous similar message [ 9418.794097] Lustre: Skipped 1 previous similar message [ 9423.638143] Lustre: server umount lustre-OST0000 complete [ 9431.520427] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9431.534780] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9431.552794] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9431.570000] Lustre: Skipped 2 previous similar messages [ 9442.277349] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9442.284415] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9442.315354] Lustre: Skipped 5 previous similar messages [ 9442.537044] Lustre: server umount lustre-MDT0000 complete [ 9446.371251] LustreError: 285958:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9446.391318] LustreError: 285958:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9447.474147] LustreError: 284787:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787124445 with bad export cookie 12430550781094694848 [ 9447.481849] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9447.495263] LustreError: 284787:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9447.966238] Lustre: server umount lustre-MDT0001 complete [ 9458.368334] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9463.518467] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9469.653046] Lustre: server umount MGS complete [ 9482.756691] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9483.509625] LustreError: 290537:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9483.570982] LustreError: 290550:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9483.698695] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9488.511585] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9488.870112] LustreError: 290538:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9498.046598] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9503.434496] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9512.725955] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9519.235984] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9527.623663] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9528.129958] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9528.140892] Lustre: Skipped 1 previous similar message [ 9534.963572] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9545.524638] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9579.492943] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9579.501842] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9579.506441] Lustre: Skipped 2 previous similar messages [ 9579.521054] Lustre: Skipped 1 previous similar message [ 9582.084675] Lustre: server umount lustre-OST0000 complete [ 9589.729769] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9589.750909] Lustre: Skipped 1 previous similar message [ 9592.926853] Lustre: server umount lustre-MDT0000 complete [ 9594.849715] LustreError: 290538:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9594.871963] LustreError: 290538:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9597.463491] LustreError: 291274:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787124595 with bad export cookie 12430550781094696675 [ 9597.469320] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9597.488636] LustreError: 291274:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9597.906959] Lustre: server umount lustre-MDT0001 complete [ 9610.862690] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [ 9614.128955] Key type lgssc unregistered [ 9614.616887] LNet: 294485:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9614.629830] LNetError: 294485:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9614.682070] LNet: Removed LNI 192.168.202.155@tcp [ 9616.119123] Key type .llcrypt unregistered [ 9616.121653] Key type ._llcrypt unregistered [ 9659.074776] Key type ._llcrypt registered [ 9659.076727] Key type .llcrypt registered [ 9659.248570] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [ 9678.838789] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [ 9679.977388] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9679.991275] alg: No test for adler32 (adler32-zlib) [ 9681.307571] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [ 9681.855744] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [ 9683.562300] Key type lgssc registered [ 9685.169035] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9725.846832] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9725.894623] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9727.226360] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9727.255757] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9727.331712] Lustre: lustre-MDT0000: new disk, initializing [ 9727.470579] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9727.483814] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9732.936835] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9745.338763] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9745.565537] Lustre: 298450:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9745.653753] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9745.664235] Lustre: Skipped 1 previous similar message [ 9745.759828] Lustre: lustre-MDT0001: new disk, initializing [ 9745.897308] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9745.968053] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9745.986650] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9750.942077] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9761.979462] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9768.561218] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9778.195513] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9778.596276] Lustre: lustre-OST0000: new disk, initializing [ 9778.603229] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9778.617099] Lustre: 300416:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9778.749808] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9787.687672] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9788.012690] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9788.032092] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9788.307311] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9801.290648] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9808.189845] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9817.932780] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9826.456669] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9836.549528] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9880.544445] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9880.545741] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9880.569282] LustreError: Skipped 1 previous similar message [ 9880.578515] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9880.591649] Lustre: Skipped 1 previous similar message [ 9880.612063] Lustre: Skipped 1 previous similar message [ 9883.548338] Lustre: server umount lustre-OST0000 complete [ 9889.251179] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9889.276268] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9889.319272] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9890.784406] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9890.792088] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9895.223956] Lustre: server umount lustre-MDT0000 complete [ 9900.632052] LustreError: 301274:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787124898 with bad export cookie 1706732610456122571 [ 9900.637715] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9900.651285] LustreError: 301274:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9901.264723] Lustre: server umount lustre-MDT0001 complete [ 9913.002542] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9918.056687] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9924.556484] Lustre: server umount MGS complete [ 9938.620945] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9939.013964] LustreError: 304295:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9939.033941] LustreError: 304295:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9939.053668] LustreError: 304308:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9939.141259] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9944.545167] LustreError: 304296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9944.728564] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9949.667706] LustreError: 304295:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9954.794865] LustreError: 304296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9955.713621] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9956.557118] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9963.775677] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [ 9974.760131] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9981.774524] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9989.407844] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9989.721314] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9995.342030] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10002.962669] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10043.360256] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10043.374273] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10043.384066] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10043.387066] Lustre: Skipped 1 previous similar message [10046.455973] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10051.558081] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10051.568481] Lustre: Skipped 2 previous similar messages [10056.161719] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10056.307530] Lustre: server umount lustre-OST0000 complete [10061.794594] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10061.799025] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10061.821125] Lustre: Skipped 2 previous similar messages [10061.832521] Lustre: Skipped 2 previous similar messages [10067.128777] Lustre: server umount lustre-MDT0000 complete [10072.035310] LustreError: 304295:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10072.084531] LustreError: 304295:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10073.722369] LustreError: 304277:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787125071 with bad export cookie 1706732610456124517 [10073.730034] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10073.735440] LustreError: 304277:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10074.530433] Lustre: server umount lustre-MDT0001 complete [10089.793992] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [10093.760527] Key type lgssc unregistered [10094.261504] LNet: 308262:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10094.268982] LNetError: 308262:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10094.292352] LNet: Removed LNI 192.168.202.155@tcp [10095.554528] Key type .llcrypt unregistered [10095.563726] Key type ._llcrypt unregistered [10121.767488] Key type ._llcrypt registered [10121.772255] Key type .llcrypt registered [10121.946689] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [10137.014809] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [10138.299283] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10138.543924] alg: No test for adler32 (adler32-zlib) [10139.649767] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [10139.903196] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [10141.575231] Key type lgssc registered [10142.926662] Lustre: Echo OBD driver; http://www.lustre.org/ [10169.199233] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [10184.525960] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10184.566695] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10185.813628] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10185.865182] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10185.945710] Lustre: lustre-MDT0000: new disk, initializing [10185.996329] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10186.009949] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10190.316962] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [10202.330292] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10202.670705] Lustre: lustre-OST0000: new disk, initializing [10202.680833] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10202.687823] Lustre: Skipped 1 previous similar message [10202.696587] Lustre: 312312:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10202.805019] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10203.339438] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10203.355968] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10203.425201] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10212.882067] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11158.599637] Lustre: server umount lustre-MDT0000 complete [11163.346477] LustreError: 311500:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787126161 with bad export cookie 7759468645408562691 [11163.355599] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11174.068470] Lustre: server umount lustre-OST0000 complete [11206.118565] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [11222.373486] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [11270.144827] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11270.554610] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11270.615185] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11270.739924] Lustre: lustre-MDT0000: new disk, initializing [11270.825639] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11270.856860] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11276.668428] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11289.667715] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11289.786764] Lustre: 316687:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11289.842261] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11289.845841] Lustre: Skipped 1 previous similar message [11289.916710] Lustre: lustre-MDT0001: new disk, initializing [11290.055153] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11290.152525] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11290.175122] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11297.518124] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11310.598737] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11318.171718] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11327.498664] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11328.022243] Lustre: lustre-OST0000: new disk, initializing [11328.030545] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11328.036091] Lustre: 318656:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11328.223871] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11335.324836] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11335.344075] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11335.504023] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11336.509837] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11348.755958] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11354.997754] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11355.322509] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11361.109879] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11361.540353] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11366.367846] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11366.375571] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11366.391246] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11366.396121] Lustre: Skipped 1 previous similar message [11366.418702] Lustre: Skipped 1 previous similar message [11370.511783] Lustre: server umount lustre-OST0000 complete [11376.611914] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11376.623879] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11376.648307] Lustre: Skipped 1 previous similar message [11376.677322] Lustre: Skipped 1 previous similar message [11381.734290] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11381.746092] Lustre: Skipped 1 previous similar message [11382.425555] Lustre: server umount lustre-MDT0000 complete [11387.151557] LustreError: 318654:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787126385 with bad export cookie 7759468645411503510 [11387.154369] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11387.167096] LustreError: 318654:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11387.872484] Lustre: server umount lustre-MDT0001 complete [11393.089584] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11395.388692] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 03:59:52 (1787126392) [11403.992808] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11404.347448] LustreError: 320972:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11404.361551] LustreError: 320972:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11404.415716] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11409.032025] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11409.892943] LustreError: 320973:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11415.018366] LustreError: 320972:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11419.745676] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11420.126685] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11424.979746] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11435.324317] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11443.251883] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11451.499139] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11452.008100] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11459.508520] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11469.535814] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11477.681965] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11477.966384] Lustre: lustre-OST0001: new disk, initializing [11477.971585] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11477.989377] Lustre: 323819:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11478.152628] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11483.758158] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11483.774489] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11483.855177] LustreError: 324113:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11484.901931] LustreError: 324113:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11484.921572] LustreError: 324113:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11485.263814] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11485.984765] LustreError: 324119:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11486.004712] LustreError: 324119:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11488.038975] LustreError: 324113:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11488.062776] LustreError: 324113:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11492.211641] LustreError: 324119:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11492.234589] LustreError: 324119:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [11497.830747] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11498.143628] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11500.490555] LustreError: 324119:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11500.513939] LustreError: 324119:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [11506.438306] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9ba9c3320000.ost_server_uuid 50 [11508.524962] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9ba9c3320000.ost_server_uuid in FULL state after 0 sec [11516.886423] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9ba9c3320000.ost_server_uuid 50 [11517.033406] LustreError: 324119:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11517.048325] LustreError: 324119:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 31 previous similar messages [11519.155741] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9ba9c3320000.ost_server_uuid in IDLE state after 0 sec [11521.320466] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11537.824265] Lustre: lustre-OST0001: Not available for connect from 192.168.202.55@tcp (stopping) [11537.903201] LustreError: lustre-OST0001-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [11537.926489] LustreError: Skipped 1 previous similar message [11537.937876] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11540.478574] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11542.909989] Lustre: lustre-OST0001: Not available for connect from 192.168.202.55@tcp (stopping) [11542.932278] Lustre: Skipped 3 previous similar messages [11544.181471] Lustre: server umount lustre-OST0001 complete [11545.579937] LustreError: 324120:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11545.613247] LustreError: 324120:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11550.701497] LustreError: 322793:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11550.741446] LustreError: 322793:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11551.712442] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11551.732000] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11551.756103] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11558.101638] Lustre: server umount lustre-OST0000 complete [11563.492342] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11563.500823] LustreError: Skipped 1 previous similar message [11563.505325] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11563.524047] Lustre: Skipped 1 previous similar message [11569.014350] Lustre: server umount lustre-MDT0000 complete [11571.168216] LustreError: 320973:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11571.194347] LustreError: 320973:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11573.253631] LustreError: 320954:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787126571 with bad export cookie 7759468645411504392 [11573.278332] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11573.291674] LustreError: 320954:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11573.908241] Lustre: server umount lustre-MDT0001 complete [11588.744489] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [11592.817486] Key type lgssc unregistered [11593.510984] LNet: 326515:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11593.522669] LNetError: 326515:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11593.563616] LNet: Removed LNI 192.168.202.155@tcp [11595.410874] Key type .llcrypt unregistered [11595.418360] Key type ._llcrypt unregistered [11633.347577] Key type ._llcrypt registered [11633.349424] Key type .llcrypt registered [11633.443701] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [11634.818481] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11634.915172] alg: No test for adler32 (adler32-zlib) [11636.014541] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [11636.256415] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [11637.975511] Key type lgssc registered [11639.272596] Lustre: Echo OBD driver; http://www.lustre.org/ [11647.670908] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11647.698936] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11649.198922] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11653.413246] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11659.747050] LustreError: 327938:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11659.775536] LustreError: 327938:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11664.589765] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11664.868984] LustreError: 327939:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11669.989825] LustreError: 327938:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11674.106417] LustreError: 327939:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11674.993190] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11675.609452] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11682.194626] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11693.498466] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11694.087471] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11695.107600] LustreError: 329621:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11695.138238] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11701.565927] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11706.341971] LustreError: 329619:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11706.354345] LustreError: 329619:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11711.254970] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11721.053299] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11721.261440] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11726.380525] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11728.372066] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [11737.389837] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11745.367066] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11745.667859] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11854.313591] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11854.332107] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11854.365042] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11854.375431] Lustre: Skipped 1 previous similar message [11859.438539] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11859.446116] Lustre: Skipped 1 previous similar message [11864.574909] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11866.080368] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11866.190687] Lustre: server umount lustre-OST0001 complete [11869.667126] LustreError: 330845:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11869.682209] LustreError: 330845:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [11874.283638] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11874.294563] LustreError: Skipped 1 previous similar message [11874.301152] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11874.312967] Lustre: Skipped 1 previous similar message [11874.317368] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11874.320878] Lustre: Skipped 1 previous similar message [11879.130979] Lustre: server umount lustre-OST0000 complete [11885.025814] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11885.035376] Lustre: Skipped 1 previous similar message [11885.040358] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11885.042921] Lustre: Skipped 1 previous similar message [11894.242379] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11894.247750] Lustre: Skipped 3 previous similar messages [11897.313951] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11897.751965] Lustre: server umount lustre-MDT0000 complete [11902.109224] LustreError: 327920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787126900 with bad export cookie 17286554690021180058 [11902.114045] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11902.125078] LustreError: 327920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11902.765965] Lustre: server umount lustre-MDT0001 complete [11917.306466] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [11920.002199] Key type lgssc unregistered [11920.247443] LNet: 332892:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11920.255066] LNetError: 332892:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11920.270302] LNet: Removed LNI 192.168.202.155@tcp [11921.129274] Key type .llcrypt unregistered [11921.132770] Key type ._llcrypt unregistered [11977.499174] Key type ._llcrypt registered [11977.501523] Key type .llcrypt registered [11977.634721] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [11979.562894] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11979.620196] alg: No test for adler32 (adler32-zlib) [11980.759714] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [11981.080954] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [11982.767323] Key type lgssc registered [11983.930896] Lustre: Echo OBD driver; http://www.lustre.org/ [11995.129085] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11995.163888] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11996.733956] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12001.595323] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12006.899873] LustreError: 335183:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12006.919869] LustreError: 335183:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12012.001540] LustreError: 335184:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12012.047292] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12012.860106] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12018.235305] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12021.459734] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12029.221675] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12029.607072] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12036.539576] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12037.819201] LustreError: 336674:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12041.959720] LustreError: 336674:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12041.979920] LustreError: 336674:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12041.993947] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12045.853676] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12046.271067] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12051.454839] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12053.843304] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12064.327926] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12073.635424] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12089.831768] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12089.849470] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12089.867158] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12092.389914] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12092.395129] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12092.419757] Lustre: Skipped 1 previous similar message [12092.440209] Lustre: Skipped 1 previous similar message [12094.372300] Lustre: server umount lustre-MDT0000 complete [12097.520037] LustreError: 335184:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12097.546154] LustreError: 335184:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12098.304524] LustreError: 335164:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787127096 with bad export cookie 15890619852571931859 [12098.344138] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12099.084916] Lustre: server umount lustre-MDT0001 complete [12112.361631] Lustre: server umount lustre-OST0000 complete [12127.291346] Lustre: server umount lustre-OST0001 complete [12147.612443] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [12159.506137] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12159.968210] LustreError: 340448:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12160.055390] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12164.846732] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12174.625639] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12175.120157] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12180.874594] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12184.154680] Lustre: 341588:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12192.547380] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12192.996853] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12199.186976] LustreError: 341943:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12199.224206] LustreError: 341943:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [12201.170652] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12204.353812] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12211.728510] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12217.376165] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12221.176805] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12230.284465] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12234.347904] Lustre: 343465:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12241.892500] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12241.900733] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12241.913980] Lustre: Skipped 1 previous similar message [12241.921633] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12241.930991] Lustre: Skipped 2 previous similar messages [12246.644249] Lustre: server umount lustre-MDT0000 complete [12248.039674] LustreError: 340444:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12248.068580] LustreError: 340444:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [12250.793936] LustreError: 340430:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787127248 with bad export cookie 15890619852571933560 [12250.811023] LustreError: 340430:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12250.824627] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12251.127096] Lustre: server umount lustre-MDT0001 complete [12265.558887] Lustre: server umount lustre-OST0000 complete [12269.536884] Lustre: 334448:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787127251/real 1787127251] req@ffff944408234700 x1873938459256064/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787127267 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12269.611843] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12269.635247] Lustre: Skipped 3 previous similar messages [12270.617339] Lustre: server umount lustre-OST0001 complete [12276.527761] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12278.877446] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12280.821732] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 04:14:37 (1787127277) [12305.389517] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [12314.773405] loop0: detected capacity change from 0 to 1099511627776 [12315.428423] SGI XFS with ACLs, security attributes, quota, no debug enabled [12315.438991] *** VALIDATE xfs *** [12315.449845] XFS (loop0): Mounting V5 Filesystem [12315.473918] XFS (loop0): Ending clean mount [12316.834292] loop1: detected capacity change from 0 to 18691697672192 [12426.818370] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [12437.578825] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [12479.211341] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12479.614270] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12479.669303] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12479.760168] Lustre: lustre-MDT0000: new disk, initializing [12479.848388] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12479.852135] Lustre: Skipped 1 previous similar message [12479.863705] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12484.980465] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12498.405398] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12498.506294] Lustre: 348601:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12498.620471] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12498.624611] Lustre: Skipped 1 previous similar message [12498.766098] Lustre: lustre-MDT0001: new disk, initializing [12498.962610] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12498.986636] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12504.332148] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12515.239197] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12522.650715] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12529.853604] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12530.233282] Lustre: lustre-OST0000: new disk, initializing [12530.238803] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12530.249224] Lustre: 350570:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12531.977213] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12531.983215] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12532.116021] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12535.953186] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12546.433963] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12551.563556] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12551.727459] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12555.773457] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12555.987099] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12557.792386] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12557.799698] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12557.811540] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12557.817628] Lustre: Skipped 5 previous similar messages [12562.914445] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12562.926036] Lustre: Skipped 1 previous similar message [12572.127425] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12572.361542] Lustre: server umount lustre-OST0000 complete [12578.278183] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12578.290203] Lustre: Skipped 2 previous similar messages [12578.291202] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12578.303278] Lustre: Skipped 3 previous similar messages [12582.843490] Lustre: server umount lustre-MDT0000 complete [12583.392651] LustreError: 349457:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12583.418522] LustreError: 349457:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12587.013201] LustreError: 348590:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787127584 with bad export cookie 15890619852571935114 [12587.016325] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12587.022588] LustreError: 348590:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12587.597466] Lustre: server umount lustre-MDT0001 complete [12596.315383] XFS (loop0): Unmounting Filesystem [12601.662616] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 04:19:58 (1787127598) [12609.439659] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12609.885309] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12609.890300] Lustre: Skipped 2 previous similar messages [12614.606173] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12624.111391] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12624.357793] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [12629.500220] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12636.857100] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12643.005208] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12650.215966] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12656.538447] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12664.977437] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12679.648969] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12679.653981] LustreError: Skipped 1 previous similar message [12679.657703] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12679.660504] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12679.663527] Lustre: Skipped 1 previous similar message [12679.683623] Lustre: Skipped 1 previous similar message [12683.517150] Lustre: server umount lustre-OST0000 complete [12694.136554] Lustre: server umount lustre-MDT0000 complete [12699.713418] LustreError: 353110:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787127697 with bad export cookie 15890619852571935996 [12699.721348] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12699.731345] LustreError: 353110:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12700.554355] Lustre: server umount lustre-MDT0001 complete [12714.772466] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [12718.592954] Key type lgssc unregistered [12719.006068] LNet: 356759:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12719.014168] LNetError: 356759:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12719.053873] LNet: Removed LNI 192.168.202.155@tcp [12720.215197] Key type .llcrypt unregistered [12720.216303] Key type ._llcrypt unregistered [12750.966457] Key type ._llcrypt registered [12750.968855] Key type .llcrypt registered [12751.061650] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [12751.836455] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12751.886794] alg: No test for adler32 (adler32-zlib) [12752.877576] Lustre: Lustre: Build Version: 2.17.56_3_g13af17f [12753.079134] LNet: Added LNI 192.168.202.155@tcp [8/256/0/180] [12754.735518] Key type lgssc registered [12755.729681] Lustre: Echo OBD driver; http://www.lustre.org/ [12762.732524] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12762.750733] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12764.155423] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12768.280121] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12774.370521] LustreError: 358328:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12774.394331] LustreError: 358328:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12776.093929] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12776.430471] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12780.520961] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12788.656577] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12794.643529] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12801.680059] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12802.088423] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12809.282825] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12820.038995] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12829.666828] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12829.706860] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12832.665704] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.55@tcp (stopping) [12832.670474] Lustre: Skipped 1 previous similar message [12834.789465] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12834.803928] Lustre: Skipped 1 previous similar message [12835.299700] Lustre: server umount lustre-MDT0001 complete [12837.747232] LustreError: 358324:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.55@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12839.912232] LustreError: 358324:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12839.941755] LustreError: 358324:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12842.871773] LustreError: 358323:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.55@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12847.994852] LustreError: 358322:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.55@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12848.024413] LustreError: 358322:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12848.072991] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12848.226915] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12848.232876] Lustre: Found index 1 for lustre-MDT0001, updating log [12848.274358] Lustre: 358315:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12848.586157] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12853.760101] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12854.543909] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [12858.378927] LustreError: 361926:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12858.395836] LustreError: 361926:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12858.411905] LustreError: 358319:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12858.618989] LustreError: 361929:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.155@tcp: cfg command failed: rc = -17 [12858.669041] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12858.669041] [13160.343678] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.55@tcp (stopping) [13160.941980] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13160.957535] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13160.975326] Lustre: Skipped 1 previous similar message [13165.445423] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.55@tcp (stopping) [13165.449220] Lustre: Skipped 2 previous similar messages [13166.374421] Lustre: server umount lustre-MDT0001 complete [13170.549019] LustreError: 358323:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.55@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13177.991207] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13178.101185] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13178.106975] Lustre: Found index 1 for lustre-MDT0001, updating log [13178.153307] Lustre: 358317:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13178.457558] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13183.508984] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13184.573889] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [13186.547613] LustreError: 370163:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13186.550469] LustreError: 370163:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13186.571826] LustreError: 358319:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13186.577805] LustreError: 358319:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13186.624260] LustreError: 370168:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13186.632319] LustreError: 370168:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13186.637958] LustreError: 370168:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.155@tcp: cfg command failed: rc = -17 [13186.646770] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.155@tcp [13186.646770] [13490.658615] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13490.670817] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13490.685965] Lustre: Skipped 1 previous similar message [13490.690377] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13490.694683] Lustre: Skipped 2 previous similar messages [13493.736258] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13496.210604] Lustre: server umount lustre-MDT0001 complete [13498.229597] LustreError: 358322:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.55@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13498.262495] LustreError: 358322:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [13507.815835] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13508.044813] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13508.058173] Lustre: Found index 1 for lustre-MDT0001, updating log [13508.148844] Lustre: 358316:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13508.511133] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13513.749497] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13514.095262] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [13517.804114] LustreError: 378273:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13517.811474] LustreError: 378273:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13517.831385] LustreError: 358319:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13517.834607] LustreError: 358319:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13517.915615] LustreError: 378276:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13517.932549] LustreError: 378276:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.155@tcp: cfg command failed: rc = -114 [13517.942804] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13517.942804] [13841.378282] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13841.381601] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13841.383616] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13841.393718] Lustre: Skipped 2 previous similar messages [13841.410786] Lustre: Skipped 5 previous similar messages [13851.103134] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13851.479413] Lustre: server umount lustre-MDT0000 complete [13851.622651] LustreError: 360952:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13851.644599] LustreError: 360952:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [13855.319043] LustreError: 358307:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787128853 with bad export cookie 1778599971301943368 [13855.323239] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13855.336139] LustreError: 358307:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13856.744600] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13861.729422] Lustre: server umount lustre-MDT0001 complete [13866.288085] Lustre: server umount lustre-OST0000 complete [13876.502964] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [13888.027413] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [13929.286617] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 04:42:06 (1787128926) [13938.062671] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13938.528036] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13938.580502] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13938.705934] Lustre: lustre-MDT0000: new disk, initializing [13938.803444] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13938.819777] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13945.424446] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [13958.313369] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13958.485490] Lustre: 388614:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13958.575116] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13958.581764] Lustre: Skipped 1 previous similar message [13958.687991] Lustre: lustre-MDT0001: new disk, initializing [13958.835745] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13958.903939] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13958.927169] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13964.419959] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [13975.487430] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13982.718745] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13991.008492] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13991.307810] Lustre: lustre-OST0000: new disk, initializing [13991.312183] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13991.319213] Lustre: 390583:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13991.430335] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13997.761277] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13997.782400] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13998.024248] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14000.307802] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_default_debug -1 all [14010.498626] Lustre: DEBUG MARKER: oleg255-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14020.078584] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [14020.095303] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14020.117611] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14020.124969] Lustre: Skipped 5 previous similar messages [14021.272430] Lustre: server umount lustre-MDT0000 complete [14023.652020] LustreError: 388621:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14023.684121] LustreError: 388621:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [14025.250456] LustreError: 388604:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787129023 with bad export cookie 1778599971301945328 [14025.259924] LustreError: MGC192.168.202.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14025.261151] LustreError: 388604:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14025.261471] LustreError: 389418:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [14025.262207] LustreError: 389418:0:(client.c:1381:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000ace65631 x1873939269025024/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [14025.685531] Lustre: server umount lustre-MDT0001 complete [14040.183636] Lustre: server umount lustre-OST0000 complete [14075.025722] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing set_hostid [14087.724124] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing load_modules_local [14127.718547] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 13810 sec ========= 04:45:24 (1787129124) [14129.323966] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 04:45:26 (1787129126) === [14133.539847] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 04:45:30 (1787129130) === [14176.252700] Lustre: DEBUG MARKER: oleg255-server.virtnet: executing unload_modules_local [14180.671683] Key type lgssc unregistered [14181.042640] LNet: 396951:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14181.052860] LNetError: 396951:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14181.087901] LNet: Removed LNI 192.168.202.155@tcp [14182.219759] Key type .llcrypt unregistered [14182.228098] Key type ._llcrypt unregistered