[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 501313741 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001014] APIC: Switch to symmetric I/O mode setup [ 0.002365] x2apic enabled [ 0.003009] Switched APIC routing to physical x2apic. [ 0.004015] kvm-guest: setup PV IPIs [ 0.006838] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008019] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010009] pid_max: default: 32768 minimum: 301 [ 0.011132] LSM: Security Framework initializing [ 0.012042] Yama: becoming mindful. [ 0.013043] SELinux: Initializing. [ 0.014074] *** VALIDATE selinux *** [ 0.024194] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.030122] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.031167] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.033137] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.034134] *** VALIDATE tmpfs *** [ 0.036020] *** VALIDATE proc *** [ 0.037281] *** VALIDATE cgroup *** [ 0.038012] *** VALIDATE cgroup2 *** [ 0.039292] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.041057] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.042009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.043031] Spectre V2 : User space: Vulnerable [ 0.044013] Speculative Store Bypass: Vulnerable [ 0.046905] debug: unmapping init [mem 0xffffffffb7459000-0xffffffffb7460fff] [ 0.050000] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.050783] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.051020] ... version: 2 [ 0.051857] ... bit width: 48 [ 0.052013] ... generic registers: 4 [ 0.052849] ... value mask: 0000ffffffffffff [ 0.053013] ... max period: 00007fffffffffff [ 0.054012] ... fixed-purpose events: 3 [ 0.055012] ... event mask: 000000070000000f [ 0.056316] rcu: Hierarchical SRCU implementation. [ 0.058517] smp: Bringing up secondary CPUs ... [ 0.059547] x86: Booting SMP configuration: [ 0.060020] .... node #0, CPUs: #1 #2 #3 [ 0.063816] smp: Brought up 1 node, 4 CPUs [ 0.065012] smpboot: Max logical packages: 1 [ 0.065981] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.146016] node 0 deferred pages initialised in 78ms [ 0.149163] devtmpfs: initialized [ 0.150292] x86/mm: Memory block size: 128MB [ 0.153064] gcov: version magic: 0x41383552 [ 0.155234] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.156081] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.157290] pinctrl core: initialized pinctrl subsystem [ 0.158226] [ 0.158881] ************************************************************* [ 0.159023] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.160016] ** ** [ 0.161014] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.162018] ** ** [ 0.163013] ** This means that this kernel is built to expose internal ** [ 0.164017] ** IOMMU data structures, which may compromise security on ** [ 0.165018] ** your system. ** [ 0.166017] ** ** [ 0.167017] ** If you see this message and you are not debugging the ** [ 0.168013] ** kernel, report this immediately to your vendor! ** [ 0.169019] ** ** [ 0.170017] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.171015] ************************************************************* [ 0.172648] NET: Registered protocol family 16 [ 0.173441] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.174064] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.175065] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.176476] cpuidle: using governor menu [ 0.178765] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.181453] PCI: Using configuration type 1 for base access [ 0.183125] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.194088] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.195026] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.197035] cryptd: max_cpu_qlen set to 1000 [ 0.200167] ACPI: Added _OSI(Module Device) [ 0.201010] ACPI: Added _OSI(Processor Device) [ 0.201902] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.202014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.205070] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.208015] ACPI: Interpreter enabled [ 0.209068] ACPI: PM: (supports S0 S3 S4 S5) [ 0.210012] ACPI: Using IOAPIC for interrupt routing [ 0.211133] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.212487] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.223111] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.224043] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.225024] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.226078] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.228056] acpiphp: Slot [2] registered [ 0.228950] acpiphp: Slot [5] registered [ 0.229079] acpiphp: Slot [6] registered [ 0.230046] acpiphp: Slot [7] registered [ 0.231042] acpiphp: Slot [8] registered [ 0.231969] acpiphp: Slot [9] registered [ 0.232080] acpiphp: Slot [10] registered [ 0.233098] acpiphp: Slot [3] registered [ 0.234046] acpiphp: Slot [4] registered [ 0.235072] acpiphp: Slot [11] registered [ 0.236058] acpiphp: Slot [12] registered [ 0.236996] acpiphp: Slot [13] registered [ 0.237097] acpiphp: Slot [14] registered [ 0.238081] acpiphp: Slot [15] registered [ 0.239011] acpiphp: Slot [16] registered [ 0.240079] acpiphp: Slot [17] registered [ 0.241109] acpiphp: Slot [18] registered [ 0.242107] acpiphp: Slot [19] registered [ 0.243203] acpiphp: Slot [20] registered [ 0.244137] acpiphp: Slot [21] registered [ 0.245142] acpiphp: Slot [22] registered [ 0.246112] acpiphp: Slot [23] registered [ 0.247112] acpiphp: Slot [24] registered [ 0.248251] acpiphp: Slot [25] registered [ 0.249114] acpiphp: Slot [26] registered [ 0.250113] acpiphp: Slot [27] registered [ 0.251124] acpiphp: Slot [28] registered [ 0.252183] acpiphp: Slot [29] registered [ 0.253243] acpiphp: Slot [30] registered [ 0.254124] acpiphp: Slot [31] registered [ 0.255073] PCI host bridge to bus 0000:00 [ 0.256032] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.257036] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.258024] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.261030] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.264024] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.266040] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.267315] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.269940] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.273402] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.282021] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.288000] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.291024] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.294025] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.297018] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.299693] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.303132] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.306056] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.310946] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.316016] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.330000] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.335014] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.341156] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.347016] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.355020] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.370021] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.378733] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.390018] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.402017] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.424021] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.440232] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.459018] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.471019] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.497020] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.510039] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.520018] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.528016] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.548021] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.564630] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.578017] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.584016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.598028] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.610315] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.620015] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.631023] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.651024] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.666197] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.668390] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.670452] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.672503] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.675268] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.680162] iommu: Default domain type: Passthrough [ 0.682605] SCSI subsystem initialized [ 0.683154] ACPI: bus type USB registered [ 0.685127] usbcore: registered new interface driver usbfs [ 0.687127] usbcore: registered new interface driver hub [ 0.688113] usbcore: registered new device driver usb [ 0.690216] pps_core: LinuxPPS API ver. 1 registered [ 0.692016] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.695107] PTP clock support registered [ 0.697049] EDAC MC: Ver: 3.0.0 [ 0.699197] PCI: Using ACPI for IRQ routing [ 0.702057] NetLabel: Initializing [ 0.703017] NetLabel: domain hash size = 128 [ 0.704013] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.706121] NetLabel: unlabeled traffic allowed by default [ 0.709061] vgaarb: loaded [ 0.710365] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.712019] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.720000] clocksource: Switched to clocksource kvm-clock [ 0.832229] VFS: Disk quotas dquot_6.6.0 [ 0.833719] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.835924] *** VALIDATE ramfs *** [ 0.837075] *** VALIDATE hugetlbfs *** [ 0.838492] pnp: PnP ACPI init [ 0.840808] pnp: PnP ACPI: found 6 devices [ 0.858070] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.861325] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.863649] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.865876] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.868567] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.871111] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.872966] NET: Registered protocol family 2 [ 0.875565] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.880702] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.883594] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.889340] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.893185] TCP: Hash tables configured (established 65536 bind 65536) [ 0.896612] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.900080] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.903065] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.906376] NET: Registered protocol family 1 [ 0.909374] RPC: Registered named UNIX socket transport module. [ 0.911400] RPC: Registered udp transport module. [ 0.913138] RPC: Registered tcp transport module. [ 0.914869] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.917483] NET: Registered protocol family 44 [ 0.919122] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.921326] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.923393] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.925783] PCI: CLS 0 bytes, default 64 [ 0.928097] Unpacking initramfs... [ 2.354334] debug: unmapping init [mem 0xffff9bf0fcc54000-0xffff9bf0fffbffff] [ 2.358167] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.360212] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.362787] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.889249] Initialise system trusted keyrings [ 2.891048] Key type blacklist registered [ 2.893172] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.904798] zbud: loaded [ 2.908534] *** VALIDATE nfs *** [ 2.909980] *** VALIDATE nfs4 *** [ 2.912485] pstore: using deflate compression [ 2.915172] Platform Keyring initialized [ 3.010937] NET: Registered protocol family 38 [ 3.012031] Key type asymmetric registered [ 3.012894] Asymmetric key parser 'x509' registered [ 3.014167] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.016125] io scheduler mq-deadline registered [ 3.017158] io scheduler kyber registered [ 3.018317] io scheduler bfq registered [ 3.019938] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.021805] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.023644] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.025483] ACPI: Power Button [PWRF] [ 3.029152] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.035616] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.047412] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.052456] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.069485] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.096195] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.124149] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.128084] Non-volatile memory driver v1.3 [ 3.129253] Linux agpgart interface v0.103 [ 3.158727] virtio_blk virtio1: [vda] 139480 512-byte logical blocks (71.4 MB/68.1 MiB) [ 3.161293] vda: detected capacity change from 0 to 71413760 [ 3.177515] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.180360] vdb: detected capacity change from 0 to 1073741824 [ 3.199643] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.202534] vdc: detected capacity change from 0 to 2621440000 [ 3.216164] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.219637] vdd: detected capacity change from 0 to 2621440000 [ 3.233593] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.236370] vde: detected capacity change from 0 to 4294967296 [ 3.250995] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.253428] vdf: detected capacity change from 0 to 4294967296 [ 3.259604] libphy: Fixed MDIO Bus: probed [ 3.268709] usbcore: registered new interface driver usbserial_generic [ 3.270236] usbserial: USB Serial support registered for generic [ 3.271857] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.275292] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.277163] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.279296] mousedev: PS/2 mouse device common for all mice [ 3.282437] rtc_cmos 00:05: RTC can wake from S4 [ 3.284461] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.285975] rtc_cmos 00:05: registered as rtc0 [ 3.291421] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.291458] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.294063] intel_pstate: CPU model not supported [ 3.299558] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.304159] hid: raw HID events driver (C) Jiri Kosina [ 3.306192] usbcore: registered new interface driver usbhid [ 3.308664] usbhid: USB HID core driver [ 3.310351] drop_monitor: Initializing network drop monitor service [ 3.313167] Initializing XFRM netlink socket [ 3.315179] NET: Registered protocol family 10 [ 3.318161] Segment Routing with IPv6 [ 3.319571] NET: Registered protocol family 17 [ 3.321616] mpls_gso: MPLS GSO support [ 3.327667] RAS: Correctable Errors collector initialized. [ 3.329248] AVX version of gcm_enc/dec engaged. [ 3.330379] AES CTR mode by8 optimization enabled [ 3.407767] sched_clock: Marking stable (3407745172, 0)->(4294269874, -886524702) [ 3.411264] registered taskstats version 1 [ 3.413192] Loading compiled-in X.509 certificates [ 3.414913] zswap: loaded using pool lzo/zbud [ 3.440079] Key type big_key registered [ 3.453415] Key type encrypted registered [ 3.455105] ima: No TPM chip found, activating TPM-bypass! [ 3.457599] ima: Allocated hash algorithm: sha1 [ 3.459243] ima: No architecture policies found [ 3.461061] evm: Initialising EVM extended attributes: [ 3.462288] evm: security.selinux [ 3.463029] evm: security.ima [ 3.463547] evm: security.capability [ 3.464272] evm: HMAC attrs: 0x1 [ 3.465929] rtc_cmos 00:05: setting system clock to 2026-06-12 13:58:00 UTC (1781272680) [ 3.470126] debug: unmapping init [mem 0xffffffffb8403000-0xffffffffb85fffff] [ 3.472018] debug: unmapping init [mem 0xffffffffb7182000-0xffffffffb7458fff] [ 3.480083] Write protecting the kernel read-only data: 28672k [ 3.482165] debug: unmapping init [mem 0xffffffffb5803000-0xffffffffb59fffff] [ 3.483704] debug: unmapping init [mem 0xffffffffb6114000-0xffffffffb61fffff] [ 3.513071] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.521877] systemd[1]: Detected virtualization kvm. [ 3.523853] systemd[1]: Detected architecture x86-64. [ 3.525819] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.552792] systemd[1]: No hostname configured. [ 3.554503] systemd[1]: Set hostname to . [ 3.556595] random: systemd: uninitialized urandom read (16 bytes read) [ 3.559320] systemd[1]: Initializing machine ID from random generator. [ 3.606166] random: ln: uninitialized urandom read (6 bytes read) [ 3.694550] random: systemd: uninitialized urandom read (16 bytes read) [ 3.697279] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 3.702137] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 3.710531] systemd[1]: Starting Setup Virtual Console... Starting Setup Virtual Console... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Initrd Root Device. [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Timers. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Slices. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.356499] device-mapper: uevent: version 1.0.3 [ 4.358476] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices.[ 5.010242] random: fast init done Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.053747] virtio_net virtio0 ens2: renamed from eth0 [ 5.159031] scsi host0: ata_piix [ 5.197449] scsi host1: ata_piix [ 5.198956] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.201479] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.322705] dracut-initqueue[585]: RTNETLINK answers: File exists [ 9.968033] random: crng init done [ 9.970437] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.248607] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.366107] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.616617] SELinux: Disabled at runtime. [ 11.677114] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.685840] systemd[1]: Detected virtualization kvm. [ 11.687848] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.196915] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.200948] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.208098] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.212153] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.215452] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.224768] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.230862] systemd[1]: Created slice User and Session Slice. [ OK ] Created slice User and Session Slice. Mounting Huge Pages File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on RPCbind Server Activation Socket. Starting Remount Root and Kernel File Systems... Mounting POSIX Message Queue File System... [ OK ] Stopped target Initrd File Systems. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target RPC Port Mapper. Activating swap /dev/disk/by-label/SWAP... Starting Apply Kernel Variables... [ OK ] Created slice system-getty.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Mounting Kernel Debug File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ 12.339356] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Reached target Paths. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on udev Control Socket. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Slices. [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Kernel Debug File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ 12.716940] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Coldplug all Devices. [ OK ] Started udev Kernel Device Manager. [ 13.066258] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.115755] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.289855] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.300751] EDAC sbridge: Ver: 1.1.2 [ 14.890473] Key type dns_resolver registered [ 15.202402] NFS: Registering the id_resolver key type [ 15.205678] Key type id_resolver registered [ 15.206835] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg135-server login: [ 63.663102] hrtimer: interrupt took 6015367 ns [ 68.830328] libcfs: loading out-of-tree module taints kernel. [ 68.957875] Key type ._llcrypt registered [ 68.959508] Key type .llcrypt registered [ 69.099316] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 92.170540] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 94.238311] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 94.252755] alg: No test for adler32 (adler32-zlib) [ 95.877653] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [ 96.662390] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 98.407720] Key type lgssc registered [ 100.243380] Lustre: Echo OBD driver; http://www.lustre.org/ [ 117.550705] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 163.556789] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 177.274244] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 177.318647] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 178.714634] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 178.779697] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 178.888613] Lustre: lustre-MDT0000: new disk, initializing [ 179.033115] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 179.075728] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 184.506565] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 199.233611] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 199.334766] Lustre: 6515:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 199.360430] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 199.363600] Lustre: Skipped 1 previous similar message [ 199.438801] Lustre: lustre-MDT0001: new disk, initializing [ 199.492273] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 199.518289] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 199.526925] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 203.998097] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 209.789401] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 220.943620] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 221.200791] Lustre: lustre-OST0000: new disk, initializing [ 221.203644] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 221.208469] Lustre: 8466:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 221.298029] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 228.432040] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 228.445316] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 228.585306] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 230.481507] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 244.983945] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 245.185801] Lustre: lustre-OST0001: new disk, initializing [ 245.190426] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 245.203111] Lustre: 9538:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 245.307979] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 251.785409] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 255.639712] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 255.653650] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 255.765049] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 264.684970] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 270.942991] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 278.454632] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing check_logdir /tmp/testlogs/ [ 284.567565] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing yml_node [ 289.714467] Lustre: DEBUG MARKER: Client: 2.17.54.4 [ 292.533209] Lustre: DEBUG MARKER: MDS: 2.17.54.4 [ 295.651560] Lustre: DEBUG MARKER: OSS: 2.17.54.4 [ 297.401768] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri Jun 12 10:02:52 EDT 2026 [ 315.370597] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 317.157609] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 337.891268] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 337.894467] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 337.902761] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 337.928512] Lustre: Skipped 1 previous similar message [ 339.458296] Lustre: server umount lustre-MDT0000 complete [ 343.326678] LustreError: 10353:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781273020 with bad export cookie 7680031711847828568 [ 343.330343] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 343.338138] LustreError: 10353:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 348.127997] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 348.129181] LustreError: 6522:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 348.158920] Lustre: Skipped 2 previous similar messages [ 348.159573] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 348.159578] Lustre: Skipped 3 previous similar messages [ 348.201549] LustreError: 6522:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 349.859889] Lustre: server umount lustre-MDT0001 complete [ 353.952431] Lustre: server umount lustre-OST0000 complete [ 358.227850] Lustre: server umount lustre-OST0001 complete [ 365.666599] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 375.949304] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 413.753531] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 413.945655] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 413.973323] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 414.050894] Lustre: lustre-MDT0000: new disk, initializing [ 414.167745] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 414.208435] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 419.652319] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 431.215948] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 431.434848] Lustre: 15091:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 431.537315] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 431.541824] Lustre: Skipped 1 previous similar message [ 431.708121] Lustre: lustre-MDT0001: new disk, initializing [ 431.871094] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 431.915437] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 431.935267] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 436.996080] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 447.254096] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 453.599988] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 460.682451] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 461.021260] Lustre: lustre-OST0000: new disk, initializing [ 461.029141] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 461.036031] Lustre: 17061:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 461.148237] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 462.430173] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 462.440179] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 462.576297] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 468.685323] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 478.961254] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 483.577541] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 483.813948] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 489.522440] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 489.778559] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 493.544092] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 493.548855] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 493.556476] Lustre: Skipped 2 previous similar messages [ 493.573106] Lustre: Skipped 1 previous similar message [ 498.757605] Lustre: server umount lustre-OST0000 complete [ 503.776277] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 503.777837] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 503.797532] Lustre: Skipped 1 previous similar message [ 503.814904] Lustre: Skipped 1 previous similar message [ 508.898381] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 508.914629] Lustre: Skipped 1 previous similar message [ 509.873415] Lustre: server umount lustre-MDT0000 complete [ 513.014829] LustreError: 15104:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 513.056584] LustreError: 15104:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 514.238209] LustreError: 15085:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781273191 with bad export cookie 7680031711847830101 [ 514.244245] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 514.253711] LustreError: 15085:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 514.686729] Lustre: server umount lustre-MDT0001 complete [ 521.022894] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 10:06:36 (1781273196) [ 528.608464] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 529.159291] LustreError: 19327:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 529.292369] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 534.515084] LustreError: 19328:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 534.805977] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 539.618182] LustreError: 19327:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 546.131339] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 552.201657] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 560.908681] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 566.763388] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 574.389995] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 581.849304] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 589.858509] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 596.763168] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 603.104190] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 603.119861] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 603.148718] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 603.372377] Lustre: server umount lustre-MDT0000 complete [ 606.710073] LustreError: 19322:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 606.755660] LustreError: 19322:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 608.173621] LustreError: 19310:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781273285 with bad export cookie 7680031711847830983 [ 608.184791] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 608.188349] LustreError: 19310:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 608.868703] Lustre: server umount lustre-MDT0001 complete [ 624.237319] Lustre: server umount lustre-OST0000 complete [ 646.355391] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 658.961913] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 659.558410] LustreError: 23710:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 659.703616] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 659.716297] Lustre: Skipped 2 previous similar messages [ 664.612454] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 673.655370] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 679.175502] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 682.826700] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 690.956540] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 699.328186] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 709.275753] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 709.451592] Lustre: lustre-OST0001: new disk, initializing [ 709.455405] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 709.463850] Lustre: 26025:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 710.851699] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 710.860140] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 710.968757] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 717.170853] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 728.196440] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 732.790734] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 751.094784] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 751.108372] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 751.139855] Lustre: Skipped 2 previous similar messages [ 751.151359] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 754.900428] Lustre: server umount lustre-MDT0000 complete [ 757.233742] LustreError: 23710:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 757.272580] LustreError: 23710:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 759.267788] LustreError: 26037:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781273436 with bad export cookie 7680031711847831830 [ 759.268804] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 759.280861] LustreError: 26037:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 759.648927] Lustre: server umount lustre-MDT0001 complete [ 775.127938] Lustre: server umount lustre-OST0000 complete [ 778.847374] Lustre: 3647:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781273439/real 1781273439] req@ffff9bf17df86d80 x1867799879471232/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781273455 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 778.863624] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 778.872605] Lustre: Skipped 3 previous similar messages [ 779.889394] Lustre: server umount lustre-OST0001 complete [ 796.977300] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 807.090625] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 807.767375] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 807.791066] Lustre: Skipped 3 previous similar messages [ 813.534744] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 822.257485] LustreError: 29048:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 822.277885] LustreError: 29048:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 823.687648] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 829.467910] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 832.774970] Lustre: 30196:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 840.035059] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 847.702369] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 857.049827] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 864.518588] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 872.343700] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 876.033878] Lustre: 32072:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 883.182140] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 883.203399] Lustre: Skipped 1 previous similar message [ 883.204776] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 883.217178] Lustre: Skipped 7 previous similar messages [ 888.954383] Lustre: server umount lustre-MDT0000 complete [ 892.714324] LustreError: 29028:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781273569 with bad export cookie 7680031711847833125 [ 892.732378] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 892.733499] LustreError: 29028:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 893.084859] Lustre: server umount lustre-MDT0001 complete [ 907.648450] Lustre: server umount lustre-OST0000 complete [ 909.793937] Lustre: 3650:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781273570/real 1781273570] req@ffff9bf17c24a300 x1867799879521792/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781273586 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 912.054678] Lustre: server umount lustre-OST0001 complete [ 920.229585] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 10:13:15 (1781273595) [ 949.241212] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 959.549983] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 960.020679] LustreError: 34875:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 960.039021] LustreError: 34875:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 964.860685] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 973.770827] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 978.585757] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 981.319241] Lustre: 36023:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 989.289408] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 996.458221] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1006.609905] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1013.239320] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1021.434951] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1025.172099] Lustre: 37903:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1032.675328] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1032.679617] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1032.708937] Lustre: Skipped 5 previous similar messages [ 1032.730411] Lustre: Skipped 7 previous similar messages [ 1037.719320] Lustre: server umount lustre-MDT0000 complete [ 1042.216800] LustreError: 36404:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781273719 with bad export cookie 7680031711847834231 [ 1042.233928] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1042.242622] LustreError: 36404:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1042.870274] Lustre: server umount lustre-MDT0001 complete [ 1058.255361] Lustre: 3649:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781273719/real 1781273719] req@ffff9bf04207ad80 x1867799879572352/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781273735 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1059.439439] Lustre: server umount lustre-OST0000 complete [ 1063.659305] Lustre: server umount lustre-OST0001 complete [ 1073.655644] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 10:15:48 (1781273748) [ 1091.565310] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 1102.833820] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1103.348623] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1103.357681] Lustre: Skipped 7 previous similar messages [ 1109.125618] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1117.645769] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1122.860943] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1125.809736] Lustre: 41197:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1132.689505] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1140.286496] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1149.643670] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1156.519881] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1163.892814] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1167.340674] Lustre: 43076:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1168.278812] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1168.282621] Lustre: Skipped 1 previous similar message [ 1174.374187] Lustre: lustre-MDT0000: Not available for connect from 192.168.201.35@tcp (stopping) [ 1174.496488] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1174.501571] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1174.515819] Lustre: Skipped 1 previous similar message [ 1177.442196] Lustre: server umount lustre-MDT0000 complete [ 1181.099764] LustreError: 40027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781273858 with bad export cookie 7680031711847835337 [ 1181.105711] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1181.121114] LustreError: 40027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1187.901766] Lustre: server umount lustre-MDT0001 complete [ 1198.627204] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1211.500992] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1216.878295] LustreError: 43950:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1216.882639] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1216.915081] LustreError: 43950:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 31 previous similar messages [ 1223.064049] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1229.304980] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1229.315561] Lustre: Skipped 1 previous similar message [ 1229.535566] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1238.621838] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1240.696159] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1247.148984] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1248.894730] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1254.369916] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1260.543469] Lustre: server umount lustre-MDT0000 complete [ 1264.826052] LustreError: 40029:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781273941 with bad export cookie 7680031711847836436 [ 1264.837689] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1264.845799] LustreError: 40029:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1265.398157] Lustre: server umount lustre-MDT0001 complete [ 1279.410748] Lustre: server umount lustre-OST0000 complete [ 1293.891923] Lustre: server umount lustre-OST0001 complete [ 1302.666734] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 10:19:38 (1781273978) [ 1320.380898] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 1333.016793] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1339.218566] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1348.842205] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1354.393418] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1357.978540] Lustre: 48837:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1365.610063] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1373.775519] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1383.166514] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1390.041072] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1398.127385] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1403.019692] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1415.653549] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1420.984114] Lustre: server umount lustre-MDT0000 complete [ 1424.897131] LustreError: 47663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781274101 with bad export cookie 7680031711847837255 [ 1424.904265] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1424.919272] LustreError: 47663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1425.400920] Lustre: server umount lustre-MDT0001 complete [ 1439.884642] Lustre: server umount lustre-OST0000 complete [ 1454.206933] Lustre: server umount lustre-OST0001 complete [ 1464.054508] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 10:22:19 (1781274139) [ 1466.636755] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1468.924302] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 10:22:23 (1781274143) [ 1480.323297] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 1488.927965] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 1511.602756] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1511.953649] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1512.066198] Lustre: lustre-MDT0000: new disk, initializing [ 1512.197340] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1517.015279] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1525.942709] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1532.296982] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1532.686609] Lustre: lustre-OST0000: new disk, initializing [ 1532.694835] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1532.704889] Lustre: Skipped 1 previous similar message [ 1532.711684] Lustre: 54916:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1533.996905] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1534.018470] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1534.066866] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1539.342730] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1549.348373] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1551.340303] Lustre: DEBUG MARKER: create test files [ 1590.243097] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1590.258965] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1590.272173] Lustre: Skipped 13 previous similar messages [ 1590.279668] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1590.286317] Lustre: Skipped 23 previous similar messages [ 1592.570909] Lustre: server umount lustre-OST0000 complete [ 1596.725912] Lustre: server umount lustre-MDT0000 complete [ 1605.678589] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 1609.242274] Key type lgssc unregistered [ 1609.649621] LNet: 56546:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1609.663496] LNetError: 56546:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1609.696138] LNet: Removed LNI 192.168.201.135@tcp [ 1611.178184] Key type .llcrypt unregistered [ 1611.182430] Key type ._llcrypt unregistered [ 1645.419385] Key type ._llcrypt registered [ 1645.421767] Key type .llcrypt registered [ 1645.556183] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 1647.549838] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1647.577641] alg: No test for adler32 (adler32-zlib) [ 1648.670848] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [ 1648.925228] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 1650.615116] Key type lgssc registered [ 1652.232847] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1660.830819] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1660.844361] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1662.403291] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1667.198789] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1675.631336] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1682.789951] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1683.298763] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1687.413799] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1687.567826] Lustre: 59285:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1687.584893] Lustre: 59285:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1687.597867] Lustre: 59285:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1687.612971] Lustre: 59285:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1687.622029] Lustre: 59285:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1687.630480] Lustre: 59285:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1690.384899] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1698.194820] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1703.834086] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1709.615675] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1718.241122] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1718.250040] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1718.261792] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1721.312273] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1726.434165] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1731.560885] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1732.063416] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1732.392297] Lustre: server umount lustre-OST0000 complete [ 1737.133873] Lustre: server umount lustre-MDT0000 complete [ 1748.985911] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 1752.038621] Key type lgssc unregistered [ 1752.325710] LNet: 60895:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1752.334780] LNetError: 60895:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1753.380872] LNet: Removed LNI 192.168.201.135@tcp [ 1754.324921] Key type .llcrypt unregistered [ 1754.328065] Key type ._llcrypt unregistered [ 1784.827580] Key type ._llcrypt registered [ 1784.829637] Key type .llcrypt registered [ 1784.974971] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 1786.678741] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1786.693268] alg: No test for adler32 (adler32-zlib) [ 1787.963241] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [ 1788.254634] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 1789.952674] Key type lgssc registered [ 1791.323027] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1799.565986] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1799.603328] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1801.212139] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1806.372744] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1814.261924] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1821.031875] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1821.432376] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1826.551748] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1826.694082] Lustre: 63729:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1826.710701] Lustre: 63729:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1826.718481] Lustre: 63729:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1826.728080] Lustre: 63729:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1826.747258] Lustre: 63729:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1826.759721] Lustre: 63729:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1828.207812] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1835.279748] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1841.676281] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1852.385050] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1852.401862] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1852.424856] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1855.891995] Lustre: server umount lustre-OST0000 complete [ 1859.385416] Lustre: server umount lustre-MDT0000 complete [ 1869.716954] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 1872.269821] Key type lgssc unregistered [ 1872.602740] LNet: 65285:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1872.608199] LNetError: 65285:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1872.631649] LNet: Removed LNI 192.168.201.135@tcp [ 1873.485615] Key type .llcrypt unregistered [ 1873.487359] Key type ._llcrypt unregistered [ 1892.108766] Key type ._llcrypt registered [ 1892.110520] Key type .llcrypt registered [ 1892.214357] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 1911.782692] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 1912.318560] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1912.339250] alg: No test for adler32 (adler32-zlib) [ 1913.318412] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [ 1913.468123] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 1915.144665] Key type lgssc registered [ 1916.168237] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1956.968612] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1957.003622] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1958.343876] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1958.386363] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1958.492107] Lustre: lustre-MDT0000: new disk, initializing [ 1958.603567] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1958.629533] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1963.484202] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1974.241853] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1974.334870] Lustre: 68466:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1974.382500] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1974.386054] Lustre: Skipped 1 previous similar message [ 1974.489707] Lustre: lustre-MDT0001: new disk, initializing [ 1974.587078] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1974.638848] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1974.662721] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1979.388664] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 1988.537600] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1994.169090] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2001.948185] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2002.253715] Lustre: lustre-OST0000: new disk, initializing [ 2002.258739] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2002.270556] Lustre: 70437:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2002.381159] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2009.413848] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2011.255750] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2011.270280] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2011.407356] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2019.400722] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2024.381871] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2024.590794] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2029.445837] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2029.625434] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2032.103266] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2032.106573] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2032.123929] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2032.138694] Lustre: Skipped 1 previous similar message [ 2037.217614] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2037.220942] Lustre: Skipped 1 previous similar message [ 2037.421151] Lustre: server umount lustre-OST0000 complete [ 2041.315456] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2041.324042] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2041.337689] Lustre: Skipped 1 previous similar message [ 2041.352218] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2042.341732] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2047.457791] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2047.465464] Lustre: Skipped 3 previous similar messages [ 2047.622691] Lustre: server umount lustre-MDT0000 complete [ 2051.191188] LustreError: 68459:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781274728 with bad export cookie 13231474302658826290 [ 2051.197512] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2051.211852] LustreError: 68459:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2051.597689] Lustre: server umount lustre-MDT0001 complete [ 2059.534376] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 10:32:15 (1781274735) [ 2069.773558] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 2108.794373] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 2119.076539] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 2152.980649] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2153.197394] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2153.248714] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2153.338560] Lustre: lustre-MDT0000: new disk, initializing [ 2153.431447] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2153.450230] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2157.941636] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2169.354945] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2169.452686] Lustre: 76465:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2169.499620] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2169.503552] Lustre: Skipped 1 previous similar message [ 2169.533919] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 2169.570410] Lustre: lustre-MDT0001: new disk, initializing [ 2169.672436] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2169.717650] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2169.731713] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2174.829959] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2183.368533] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2190.200566] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2199.178317] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2199.404660] Lustre: lustre-OST0000: new disk, initializing [ 2199.407366] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2199.411773] Lustre: 78435:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2199.459697] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2205.790289] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2205.810164] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2206.028274] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2207.374065] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2216.742559] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2223.047516] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2223.388551] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2228.507191] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2228.742866] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2231.778118] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2231.785968] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2231.790234] LustreError: Skipped 1 previous similar message [ 2231.791025] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2231.820638] Lustre: Skipped 1 previous similar message [ 2237.629480] Lustre: server umount lustre-OST0000 complete [ 2246.624742] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2246.644065] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2249.024435] Lustre: server umount lustre-MDT0000 complete [ 2254.939131] LustreError: 76457:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781274931 with bad export cookie 13231474302658827165 [ 2254.946966] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2254.965192] LustreError: 76457:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2255.455927] Lustre: server umount lustre-MDT0001 complete [ 2266.155168] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 10:35:41 (1781274941) [ 2273.981633] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2274.513979] LustreError: 80847:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2274.537337] LustreError: 80847:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2274.615366] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2279.844791] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2279.910886] LustreError: 80848:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2284.040464] LustreError: 80847:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2288.865457] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2293.851284] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2301.917348] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2308.714968] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2315.518636] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2316.001741] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2316.004587] Lustre: Skipped 1 previous similar message [ 2323.117373] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2331.666935] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2335.686862] Lustre: 83522:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 6079e20e-bcdc-4252-9e41-78fccc7e3611 at adminstrative request [ 2339.813955] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2339.850311] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2349.024439] Lustre: MGS: Client 6079e20e-bcdc-4252-9e41-78fccc7e3611 (at 0@lo) reconnecting [ 2349.042093] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xb79fa3c660b05716 to 0xb79fa3c660b059b6 [ 2349.056220] Lustre: MGC192.168.201.135@tcp: Connection restored to 0@lo (at 0@lo) [ 2374.208700] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2374.444851] Lustre: lustre-OST0001: new disk, initializing [ 2374.449945] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2374.463340] Lustre: 83914:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2381.183551] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2382.941040] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2382.953165] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2383.114869] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2392.046199] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2398.689907] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2398.704369] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2398.723771] Lustre: Skipped 1 previous similar message [ 2398.732785] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2398.735483] Lustre: Skipped 6 previous similar messages [ 2401.799963] Lustre: server umount lustre-OST0001 complete [ 2403.818937] LustreError: 82670:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2403.830133] LustreError: 82670:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2408.955259] LustreError: 82669:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2408.984921] LustreError: 82669:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2411.404837] Lustre: server umount lustre-OST0000 complete [ 2417.124142] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2417.135262] LustreError: Skipped 3 previous similar messages [ 2417.141563] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2417.169680] Lustre: Skipped 3 previous similar messages [ 2421.101922] Lustre: server umount lustre-MDT0000 complete [ 2424.291461] LustreError: 82623:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2424.304317] LustreError: 82623:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2424.607041] LustreError: 80827:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781275101 with bad export cookie 13231474302658828726 [ 2424.617918] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2424.930775] Lustre: server umount lustre-MDT0001 complete [ 2432.826349] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 10:38:28 (1781275108) [ 2434.437385] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2436.385485] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 10:38:31 (1781275111) [ 2437.962704] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2439.992396] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 10:38:35 (1781275115) [ 2441.755377] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2443.412591] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 10:38:38 (1781275118) [ 2445.236959] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2447.110192] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 10:38:42 (1781275122) [ 2449.064645] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2455.437279] loop: module loaded [ 2455.477367] loop0: detected capacity change from 0 to 4294967296 [ 2455.557832] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2457.150492] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2460.411730] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2464.415288] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2490.344138] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2490.416600] LustreError: 88036:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2490.427323] LustreError: 88036:0:(obd_mount.c:250:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2490.438374] LustreError: 88036:0:(tgt_mount.c:2534:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2490.446140] LustreError: 88036:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2512.279721] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 2524.961978] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 2569.755269] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2570.339407] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2570.373761] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2570.475386] Lustre: lustre-MDT0000: new disk, initializing [ 2570.605263] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2570.610192] Lustre: Skipped 1 previous similar message [ 2570.632888] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2576.646896] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2590.409516] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2590.609518] Lustre: 90905:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2590.913174] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2597.678386] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2608.969904] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2615.486119] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2623.596678] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2624.098331] Lustre: 92875:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2631.874901] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2633.271657] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2633.430512] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2641.950978] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2647.219732] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2647.462485] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2653.072293] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2653.298878] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2659.304929] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2659.305233] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2659.328102] LustreError: Skipped 1 previous similar message [ 2659.333310] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2659.354886] Lustre: Skipped 2 previous similar messages [ 2659.380244] Lustre: Skipped 9 previous similar messages [ 2661.690283] Lustre: server umount lustre-OST0000 complete [ 2680.287275] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2680.647439] Lustre: server umount lustre-MDT0000 complete [ 2684.908724] LustreError: 90912:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2684.955621] LustreError: 90912:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2685.774338] LustreError: 90896:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781275362 with bad export cookie 13231474302658829342 [ 2685.776724] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2685.792897] LustreError: 90896:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2686.653195] Lustre: server umount lustre-MDT0001 complete [ 2697.656785] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 10:42:52 (1781275372) [ 2705.936184] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2706.862219] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 2706.878626] Lustre: Skipped 2 previous similar messages [ 2713.786475] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2721.249531] LustreError: 95291:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2721.258269] LustreError: 95291:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2722.939784] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2727.508941] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2736.448705] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2743.302194] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2751.961880] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2759.088543] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2768.515649] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2774.012742] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2776.540887] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 2781.106433] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2781.409609] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2789.062136] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2789.157583] Lustre: lustre-OST0001: new disk, initializing [ 2789.158921] Lustre: Skipped 2 previous similar messages [ 2789.160512] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2789.162098] Lustre: Skipped 3 previous similar messages [ 2789.164250] Lustre: 98678:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2796.697032] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2797.262601] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2797.279113] Lustre: Skipped 2 previous similar messages [ 2797.286625] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2797.392787] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2805.562738] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2811.691683] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2811.917097] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2817.211607] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2817.509844] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2838.989510] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2863.110356] Lustre: Failing over lustre-MDT0000 [ 2864.102220] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2864.109629] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2864.125629] Lustre: Skipped 4 previous similar messages [ 2864.136035] Lustre: Skipped 10 previous similar messages [ 2865.687398] Lustre: server umount lustre-MDT0000 complete [ 2866.019527] LustreError: 95286:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.35@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2866.032938] LustreError: 95286:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2885.602204] Lustre: 66692:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781275546/real 1781275546] req@00000000f49e68e2 x1867801784766080/t0(0) o400->MGC192.168.201.135@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1781275562 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2885.639127] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2891.543927] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2891.548531] LDISKFS-fs (dm-0): recovery complete [ 2891.554231] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2896.217295] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2897.735360] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2897.735751] Lustre: 100893:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2901.385482] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 2901.513496] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2901.515984] Lustre: 95291:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2901.559735] Lustre: 95291:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 1 previous similar message [ 2901.707918] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2902.038074] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2902.378498] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2902.400505] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 2902.737309] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2902.749982] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 2902.766983] Lustre: 101527:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2902.785478] Lustre: 101527:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 7 previous similar messages [ 2903.430759] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2903.446622] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 2 previous similar messages [ 2903.767728] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2903.772843] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 2 previous similar messages [ 2905.066988] Lustre: 101527:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2905.074979] Lustre: 101527:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 6 previous similar messages [ 2905.714616] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2905.735825] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 6 previous similar messages [ 2906.063145] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2906.080752] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 6 previous similar messages [ 2909.256163] Lustre: 101527:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2909.272635] Lustre: 101527:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 12 previous similar messages [ 2909.922972] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2909.935380] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 2910.271602] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2910.282984] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 2917.279330] Lustre: 101527:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2917.289516] Lustre: 101527:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 24 previous similar messages [ 2917.946099] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2917.958474] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 24 previous similar messages [ 2918.288039] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2918.298898] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 24 previous similar messages [ 2933.522366] Lustre: 101527:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2933.547200] Lustre: 101527:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 50 previous similar messages [ 2934.216198] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2934.241038] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 50 previous similar messages [ 2934.559118] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2934.575538] LustreError: 101527:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 50 previous similar messages [ 2957.167175] Lustre: 101527:0:(ldlm_lib.c:2380:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2957.171518] Lustre: 101527:0:(ldlm_lib.c:2390:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2957.187846] Lustre: 101527:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client ac19df96-d835-431c-8528-2e7357999f7f@192.168.201.35@tcp [ 2957.216827] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2957.230982] Lustre: 101527:0:(ldlm_lib.c:1900:abort_req_replay_queue()) @@@ aborted: req@0000000045615c70 x1867801765678208/t0(8589934770) o35->ac19df96-d835-431c-8528-2e7357999f7f@192.168.201.35@tcp:369/0 lens 392/0 e 0 to 0 dl 1781275644 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2957.273453] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2957.302442] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2957.310219] Lustre: lustre-MDT0000: Denying connection for new client ac19df96-d835-431c-8528-2e7357999f7f (at 192.168.201.35@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2957.345117] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2957.347622] Lustre: Skipped 2 previous similar messages [ 2957.376276] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2957.430104] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 2957.443438] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2982.445140] Lustre: server umount lustre-OST0000 complete [ 2986.976364] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2986.994950] LustreError: Skipped 1 previous similar message [ 2992.913330] Lustre: server umount lustre-OST0001 complete [ 2998.241797] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2998.250469] Lustre: Skipped 5 previous similar messages [ 2998.255541] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2998.261359] Lustre: Skipped 5 previous similar messages [ 3011.041437] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3011.290821] Lustre: server umount lustre-MDT0000 complete [ 3013.088624] LustreError: 95291:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3013.118421] LustreError: 95291:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 45 previous similar messages [ 3015.713935] LustreError: 95272:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781275692 with bad export cookie 13231474302658880561 [ 3015.721906] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3015.722748] LustreError: 95272:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3016.126858] Lustre: server umount lustre-MDT0001 complete [ 3024.790923] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 10:48:19 (1781275699) [ 3032.550036] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3032.563824] Lustre: *** cfs_fail_loc=197, val=0*** [ 3048.288626] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3053.408767] LustreError: 103504:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.135@tcp: failed processing log, type 4: rc = -110 [ 3058.583523] Lustre: *** cfs_fail_loc=197, val=0*** [ 3058.592639] Lustre: Skipped 65 previous similar messages [ 3079.138114] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3079.147042] Lustre: Skipped 4 previous similar messages [ 3086.278474] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3097.173959] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3099.228736] Lustre: server umount lustre-OST0000 complete [ 3109.148620] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 10:49:43 (1781275783) [ 3130.693913] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3136.459356] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3147.016609] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3151.110358] LustreError: 105155:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3151.118291] LustreError: 105155:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3151.124731] LustreError: 105155:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 20, retries 0, failed: rc = -5 [ 3151.555412] Lustre: server umount lustre-MDT0000 complete [ 3189.305439] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 3200.078792] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 3243.574926] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3243.780830] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3243.809863] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3243.869186] Lustre: lustre-MDT0000: new disk, initializing [ 3244.113880] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3249.395620] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3260.569346] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3260.714698] Lustre: 109331:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3260.954626] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3266.302702] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3276.692488] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3284.274637] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3289.584115] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3289.916073] Lustre: 111300:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3297.044388] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3297.559400] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3308.347810] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3313.421676] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3313.632652] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3319.050220] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3319.254445] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3323.362834] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3323.378168] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3323.389097] LustreError: Skipped 2 previous similar messages [ 3323.390383] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3323.390391] Lustre: Skipped 5 previous similar messages [ 3323.434579] Lustre: Skipped 2 previous similar messages [ 3327.641913] Lustre: server umount lustre-OST0000 complete [ 3338.161294] Lustre: server umount lustre-MDT0000 complete [ 3338.725458] LustreError: 109344:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3338.739851] LustreError: 109344:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 3342.151424] LustreError: 109325:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781276019 with bad export cookie 13231474302658891152 [ 3342.162733] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3342.168879] LustreError: 109325:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3342.570607] Lustre: server umount lustre-MDT0001 complete [ 3352.036732] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 10:53:47 (1781276027) [ 3362.080490] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 3365.079249] Key type lgssc unregistered [ 3365.481678] LNet: 113793:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3365.490595] LNetError: 113793:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3365.510808] LNet: Removed LNI 192.168.201.135@tcp [ 3366.493374] Key type .llcrypt unregistered [ 3366.500082] Key type ._llcrypt unregistered [ 3387.401899] Key type ._llcrypt registered [ 3387.404251] Key type .llcrypt registered [ 3387.547823] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 3413.012121] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 3413.998952] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3414.120447] alg: No test for adler32 (adler32-zlib) [ 3415.419214] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [ 3415.882694] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 3417.615208] Key type lgssc registered [ 3418.957768] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3467.541987] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3467.579923] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3469.027471] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3469.050806] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3469.093469] Lustre: lustre-MDT0000: new disk, initializing [ 3469.128988] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3469.137724] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3473.972542] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3484.549034] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3484.888515] Lustre: lustre-OST0000: new disk, initializing [ 3484.893437] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3484.911044] Lustre: Skipped 1 previous similar message [ 3484.916989] Lustre: 117969:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3485.008451] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3491.209991] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3493.491640] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3493.503828] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3493.557773] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3502.419899] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3502.615433] Lustre: lustre-OST0001: new disk, initializing [ 3502.628435] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3502.636928] Lustre: 118832:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3502.753237] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3509.949735] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3511.047897] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3511.057756] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3511.084388] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3521.510708] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3521.529042] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3521.537140] Lustre: Skipped 1 previous similar message [ 3526.077271] Lustre: server umount lustre-MDT0000 complete [ 3533.458264] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3538.649327] LustreError: 117156:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781276215 with bad export cookie 3097220907205970017 [ 3538.653512] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3538.666851] LustreError: 117156:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3538.999815] Lustre: server umount lustre-OST0000 complete [ 3543.080718] Lustre: server umount lustre-OST0001 complete [ 3551.802236] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 10:57:07 (1781276227) [ 3561.298108] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 3584.603758] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 10:57:40 (1781276260) [ 3625.898737] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 3635.031264] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 3675.930155] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3676.168794] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3676.197756] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3676.271728] Lustre: lustre-MDT0000: new disk, initializing [ 3676.382973] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3676.404637] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3681.428489] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3692.568687] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3692.689465] Lustre: 125581:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3692.784378] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3692.796516] Lustre: Skipped 1 previous similar message [ 3692.900595] Lustre: lustre-MDT0001: new disk, initializing [ 3693.064677] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3693.107743] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3693.129035] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3698.008207] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3706.415153] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3711.737932] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3720.159556] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3720.461598] Lustre: lustre-OST0000: new disk, initializing [ 3720.464930] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3720.476633] Lustre: 127547:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3720.549775] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3725.947339] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3726.491733] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3726.504972] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3726.664287] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3735.236263] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3740.963590] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3741.340828] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3748.059616] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3748.550897] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3752.430972] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3752.434093] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3752.440790] LustreError: Skipped 1 previous similar message [ 3752.446224] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3752.461442] Lustre: Skipped 2 previous similar messages [ 3752.497307] Lustre: Skipped 1 previous similar message [ 3757.754742] Lustre: server umount lustre-OST0000 complete [ 3764.717039] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3764.740195] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3764.775271] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3767.776870] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3767.778189] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3767.821463] Lustre: Skipped 1 previous similar message [ 3769.663338] Lustre: server umount lustre-MDT0000 complete [ 3772.897123] LustreError: 126940:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3772.920795] LustreError: 126940:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 3774.202920] LustreError: 125572:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781276451 with bad export cookie 3097220907205971235 [ 3774.205978] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3774.212174] LustreError: 125572:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3774.495670] Lustre: server umount lustre-MDT0001 complete [ 3783.979179] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 11:00:59 (1781276459) [ 3792.079750] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3792.724280] LustreError: 129973:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3792.811349] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3797.961719] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3797.984290] LustreError: 129974:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3802.112350] LustreError: 129973:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3806.774927] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3811.957837] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3820.086622] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3825.636482] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3832.493140] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3832.805408] LustreError: 131797:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3833.059094] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3833.066251] Lustre: Skipped 1 previous similar message [ 3839.897229] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 3847.486492] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3852.357778] Lustre: *** cfs_fail_loc=159, val=0*** [ 3868.556539] Lustre: lustre-MDT0000: Client 326832be-2d6f-4388-8f1b-6b0f1ed4777d (at 192.168.201.35@tcp) reconnecting [ 3870.420738] Lustre: *** cfs_fail_loc=159, val=0*** [ 3870.422957] Lustre: Skipped 5 previous similar messages [ 3887.064259] Lustre: lustre-MDT0000: Client 326832be-2d6f-4388-8f1b-6b0f1ed4777d (at 192.168.201.35@tcp) reconnecting [ 3909.095502] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3909.114211] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3909.118338] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3909.133996] Lustre: Skipped 1 previous similar message [ 3910.283951] Lustre: server umount lustre-OST0000 complete [ 3919.342545] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3919.370402] Lustre: Skipped 1 previous similar message [ 3919.374849] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3921.493953] Lustre: server umount lustre-MDT0000 complete [ 3924.449491] LustreError: 131553:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3924.478454] LustreError: 131553:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3926.237189] LustreError: 132574:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781276603 with bad export cookie 3097220907205972117 [ 3926.246927] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3926.255448] LustreError: 132574:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3927.304068] Lustre: server umount lustre-MDT0001 complete [ 3947.683761] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 3951.227697] Key type lgssc unregistered [ 3951.639387] LNet: 133774:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3951.662145] LNetError: 133774:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3951.693532] LNet: Removed LNI 192.168.201.135@tcp [ 3952.814995] Key type .llcrypt unregistered [ 3952.817587] Key type ._llcrypt unregistered [ 3978.188597] Key type ._llcrypt registered [ 3978.190954] Key type .llcrypt registered [ 3978.327575] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 3979.854862] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3979.989090] alg: No test for adler32 (adler32-zlib) [ 3981.605335] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [ 3981.970025] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 3983.703300] Key type lgssc registered [ 3985.507690] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3994.579585] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3994.601756] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3996.072636] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4000.949512] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4006.368371] LustreError: 135198:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4006.397983] LustreError: 135198:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4009.103514] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4009.423496] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4015.456591] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4023.868120] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4028.736532] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4037.057836] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4037.645279] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4041.824632] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4046.977344] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4056.354969] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4064.771364] Lustre: *** cfs_fail_loc=159, val=0*** [ 4081.022588] Lustre: lustre-MDT0000: Client d1cae596-6e65-4757-bff8-32e9bf2e29e9 (at 192.168.201.35@tcp) reconnecting [ 4083.613308] Lustre: *** cfs_fail_loc=159, val=0*** [ 4100.006461] Lustre: lustre-MDT0001: Client d1cae596-6e65-4757-bff8-32e9bf2e29e9 (at 192.168.201.35@tcp) reconnecting [ 4101.808561] Lustre: *** cfs_fail_loc=159, val=0*** [ 4101.810369] Lustre: Skipped 3 previous similar messages [ 4117.369165] Lustre: lustre-MDT0001: Client d1cae596-6e65-4757-bff8-32e9bf2e29e9 (at 192.168.201.35@tcp) reconnecting [ 4134.616865] Lustre: 138267:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4137.589429] Lustre: *** cfs_fail_loc=159, val=0*** [ 4137.595156] Lustre: Skipped 3 previous similar messages [ 4153.278648] Lustre: lustre-MDT0000: Client 60b34642-6ff0-4804-bd4a-b95f9de212c7 (at 192.168.201.35@tcp) reconnecting [ 4155.438672] Lustre: *** cfs_fail_loc=159, val=0*** [ 4155.445345] Lustre: Skipped 14 previous similar messages [ 4171.642788] Lustre: lustre-MDT0000: Client 60b34642-6ff0-4804-bd4a-b95f9de212c7 (at 192.168.201.35@tcp) reconnecting [ 4173.614627] Lustre: 138558:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4193.762843] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4193.767655] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4193.783844] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4195.808301] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4195.812120] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4195.824651] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4196.835633] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4196.844772] Lustre: Skipped 1 previous similar message [ 4198.503729] Lustre: server umount lustre-OST0000 complete [ 4204.001325] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4204.020820] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4204.047898] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4207.085641] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4212.192759] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4212.199687] Lustre: Skipped 3 previous similar messages [ 4217.313456] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4217.470712] Lustre: server umount lustre-MDT0000 complete [ 4221.422870] LustreError: 135199:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4221.452217] LustreError: 135199:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4221.696938] LustreError: 135177:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781276898 with bad export cookie 373665760875444346 [ 4221.699937] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4221.710276] LustreError: 135177:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4222.019749] Lustre: server umount lustre-MDT0001 complete [ 4233.010695] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 4236.919043] Key type lgssc unregistered [ 4237.323481] LNet: 139639:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4237.332602] LNetError: 139639:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4237.366881] LNet: Removed LNI 192.168.201.135@tcp [ 4238.835476] Key type .llcrypt unregistered [ 4238.837726] Key type ._llcrypt unregistered [ 4266.365434] Key type ._llcrypt registered [ 4266.366489] Key type .llcrypt registered [ 4266.426813] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 4267.585774] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4267.597753] alg: No test for adler32 (adler32-zlib) [ 4268.548606] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [ 4268.667683] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 4270.271164] Key type lgssc registered [ 4271.006724] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4278.639887] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4278.684751] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4280.293818] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4285.246954] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4290.537514] LustreError: 141054:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4290.557476] LustreError: 141054:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4295.291349] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4295.610090] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4299.974694] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4307.622970] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4313.124646] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4320.019817] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4320.241222] LustreError: 142877:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4320.599103] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4325.704573] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4325.732662] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4326.830347] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4333.356531] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4338.265849] Lustre: 143634:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4340.906317] Lustre: 143683:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4342.122836] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4342.965812] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4346.767768] Lustre: 143828:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4347.649548] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4348.459698] Lustre: 143927:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4356.604432] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4356.612367] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4356.625162] LustreError: Skipped 1 previous similar message [ 4356.629070] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4356.642567] Lustre: Skipped 1 previous similar message [ 4356.666117] Lustre: Skipped 1 previous similar message [ 4358.726739] Lustre: server umount lustre-OST0000 complete [ 4366.817337] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4366.828535] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4366.829874] Lustre: Skipped 1 previous similar message [ 4366.850644] Lustre: Skipped 1 previous similar message [ 4368.795599] Lustre: server umount lustre-MDT0000 complete [ 4371.942589] LustreError: 142074:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4371.963070] LustreError: 142074:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4372.760208] LustreError: 141034:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781277049 with bad export cookie 2312826820061102881 [ 4372.780431] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4372.785455] LustreError: 141034:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4373.062897] Lustre: server umount lustre-MDT0001 complete [ 4385.231650] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 4388.740775] Key type lgssc unregistered [ 4388.975531] LNet: 144989:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4388.983901] LNetError: 144989:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4388.998407] LNet: Removed LNI 192.168.201.135@tcp [ 4389.924990] Key type .llcrypt unregistered [ 4389.926595] Key type ._llcrypt unregistered [ 4422.539243] Key type ._llcrypt registered [ 4422.543279] Key type .llcrypt registered [ 4422.729636] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 4425.210625] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4425.300655] alg: No test for adler32 (adler32-zlib) [ 4426.814872] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [ 4427.118090] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 4428.951930] Key type lgssc registered [ 4430.671434] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4439.477141] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4439.495749] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4441.126716] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4446.431494] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4451.309167] LustreError: 146415:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4451.339775] LustreError: 146415:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4456.416219] LustreError: 146414:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4456.906357] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4457.187248] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4460.942696] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4470.238738] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4478.001664] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4487.461887] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4487.980689] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4495.343924] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4502.573995] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4503.371523] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4511.535789] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4519.157742] Lustre: *** cfs_fail_loc=159, val=0*** [ 4535.671124] Lustre: lustre-MDT0000: Client 0f93e6de-82b5-4a08-bc0f-16f34fb2862c (at 192.168.201.35@tcp) reconnecting [ 4540.901687] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4540.904586] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4540.913447] LustreError: Skipped 1 previous similar message [ 4540.948784] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4542.435402] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4542.439239] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4542.647636] Lustre: server umount lustre-OST0000 complete [ 4547.559889] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4547.573280] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4547.578048] Lustre: Skipped 1 previous similar message [ 4552.688923] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4553.076258] Lustre: server umount lustre-MDT0000 complete [ 4557.215197] LustreError: 146394:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781277234 with bad export cookie 16599467152839915093 [ 4557.220797] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4557.244783] LustreError: 146394:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4557.767517] Lustre: server umount lustre-MDT0001 complete [ 4570.696642] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 4574.016511] Key type lgssc unregistered [ 4574.358810] LNet: 150106:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4574.371676] LNetError: 150106:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4574.394939] LNet: Removed LNI 192.168.201.135@tcp [ 4575.513942] Key type .llcrypt unregistered [ 4575.518400] Key type ._llcrypt unregistered [ 4601.001414] Key type ._llcrypt registered [ 4601.004099] Key type .llcrypt registered [ 4601.096456] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 4602.719260] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4602.748498] alg: No test for adler32 (adler32-zlib) [ 4603.892285] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [ 4604.103063] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 4605.775167] Key type lgssc registered [ 4607.315690] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4615.830484] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4615.840610] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4617.360728] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4622.575791] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4627.942017] LustreError: 151524:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4627.972326] LustreError: 151524:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4631.449664] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4632.023763] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4636.373672] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4643.073981] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4648.253234] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4654.928221] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4655.266225] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4656.305309] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4661.703855] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4662.455403] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4668.390975] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4687.888804] Lustre: lustre-OST0000: evicting aaf1dac2-07ef-499b-b350-15908e56079e (at 192.168.201.35@tcp) by administrative request [ 4687.903469] Lustre: Skipped 1 previous similar message [ 4718.051238] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4718.063854] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4718.087458] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4719.075359] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4719.082332] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4719.099762] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4719.106523] Lustre: Skipped 1 previous similar message [ 4722.721630] Lustre: server umount lustre-OST0000 complete [ 4729.314880] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4729.321704] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4729.326454] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4729.353017] Lustre: Skipped 1 previous similar message [ 4732.954498] Lustre: server umount lustre-MDT0000 complete [ 4734.433334] LustreError: 151523:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4734.450724] LustreError: 151523:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4736.623156] LustreError: 152264:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781277413 with bad export cookie 14589260433239738540 [ 4736.631034] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4736.946703] Lustre: server umount lustre-MDT0001 complete [ 4748.652206] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 4751.047747] Key type lgssc unregistered [ 4751.295508] LNet: 155587:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4751.303571] LNetError: 155587:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4751.318617] LNet: Removed LNI 192.168.201.135@tcp [ 4751.914375] Key type .llcrypt unregistered [ 4751.916177] Key type ._llcrypt unregistered [ 4818.344569] Key type ._llcrypt registered [ 4818.347968] Key type .llcrypt registered [ 4818.468748] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 4819.569438] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4819.591103] alg: No test for adler32 (adler32-zlib) [ 4820.976079] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [ 4821.312835] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 4823.047191] Key type lgssc registered [ 4824.582071] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4832.615234] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4832.669894] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4834.709246] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4840.765122] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4844.018391] LustreError: 158076:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4844.051754] LustreError: 158076:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4849.135474] LustreError: 158077:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4849.469642] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4849.909680] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4853.208722] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4859.723819] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4865.712116] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4872.793554] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4873.193185] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4874.248755] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4879.411680] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4880.326644] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 4887.899389] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4899.815218] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4899.815863] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4899.819474] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4899.823987] LustreError: Skipped 1 previous similar message [ 4899.853277] Lustre: Skipped 1 previous similar message [ 4904.946451] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4904.954590] Lustre: Skipped 2 previous similar messages [ 4910.051642] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4911.071549] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4911.633987] Lustre: server umount lustre-OST0000 complete [ 4916.704638] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4916.719243] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4916.745066] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4916.761338] Lustre: Skipped 1 previous similar message [ 4920.291294] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4921.869597] Lustre: server umount lustre-MDT0000 complete [ 4925.411318] LustreError: 158076:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4925.438733] LustreError: 158076:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4925.695653] LustreError: 158056:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781277602 with bad export cookie 6845420005233214499 [ 4925.705295] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4925.706326] LustreError: 158056:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4926.163813] Lustre: server umount lustre-MDT0001 complete [ 4937.908508] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 4940.633655] Key type lgssc unregistered [ 4940.969239] LNet: 161621:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4940.981766] LNetError: 161621:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4940.995912] LNet: Removed LNI 192.168.201.135@tcp [ 4941.951688] Key type .llcrypt unregistered [ 4941.956908] Key type ._llcrypt unregistered [ 4971.468559] Key type ._llcrypt registered [ 4971.471197] Key type .llcrypt registered [ 4971.655283] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 4973.591975] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4973.702239] alg: No test for adler32 (adler32-zlib) [ 4974.648488] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [ 4974.794114] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 4976.399953] Key type lgssc registered [ 4977.347249] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4991.476606] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 11:21:06 (1781277666) [ 5008.005329] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 5018.649901] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 5052.531100] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5052.606628] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5053.942330] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5054.011841] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5054.106726] Lustre: lustre-MDT0000: new disk, initializing [ 5054.221520] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5054.251584] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5059.172461] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5069.657667] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5069.780710] Lustre: 165885:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5069.838793] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5069.842233] Lustre: Skipped 1 previous similar message [ 5069.846771] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 5069.910175] Lustre: lustre-MDT0001: new disk, initializing [ 5070.005993] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5070.136928] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5070.146635] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5074.341587] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5083.102410] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5088.334181] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5095.561086] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5095.901459] Lustre: lustre-OST0000: new disk, initializing [ 5095.911515] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5095.920634] Lustre: 167853:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5096.003530] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5102.572541] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5104.185180] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5104.199601] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5104.377039] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5111.425587] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5125.096591] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5125.097145] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5125.103412] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5125.123646] Lustre: Skipped 1 previous similar message [ 5128.166995] Lustre: server umount lustre-OST0000 complete [ 5131.750052] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5131.759574] LustreError: Skipped 1 previous similar message [ 5131.767500] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5131.794205] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5131.806777] Lustre: Skipped 1 previous similar message [ 5135.330132] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5135.331573] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5135.365950] Lustre: Skipped 1 previous similar message [ 5138.159186] Lustre: server umount lustre-MDT0000 complete [ 5142.010176] LustreError: 165875:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781277819 with bad export cookie 4016606450992268277 [ 5142.018804] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5142.023336] LustreError: 165875:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5142.576378] Lustre: server umount lustre-MDT0001 complete [ 5154.126609] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 5156.221837] Key type lgssc unregistered [ 5156.454387] LNet: 169759:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5156.465304] LNetError: 169759:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5156.486979] LNet: Removed LNI 192.168.201.135@tcp [ 5157.340284] Key type .llcrypt unregistered [ 5157.342260] Key type ._llcrypt unregistered [ 5181.116666] Key type ._llcrypt registered [ 5181.120678] Key type .llcrypt registered [ 5181.349652] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 5182.937339] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5183.097876] alg: No test for adler32 (adler32-zlib) [ 5184.304257] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [ 5184.651791] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 5186.327208] Key type lgssc registered [ 5187.651764] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5196.013288] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5196.020783] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5197.837346] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5203.571858] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5207.014450] LustreError: 171173:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5207.029200] LustreError: 171173:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5212.138836] LustreError: 171174:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5212.795278] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5213.073994] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5217.805232] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5225.848655] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5232.797842] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5239.573450] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5239.850376] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5242.930627] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5246.279845] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 5254.119778] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5264.400780] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5264.445201] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5264.464302] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5264.476261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.498860] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5265.587691] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5265.620844] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5265.649286] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5265.653397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.674198] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5266.615945] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5266.626711] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5266.635675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.575775] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5267.582701] Lustre: Skipped 1 previous similar message [ 5267.586554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.601306] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5267.612946] Lustre: Skipped 1 previous similar message [ 5268.387293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.342370] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5269.345759] Lustre: Skipped 2 previous similar messages [ 5269.353974] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5269.358658] Lustre: Skipped 2 previous similar messages [ 5269.361844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.492229] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5270.498686] Lustre: Skipped 2 previous similar messages [ 5270.505713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.528288] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5270.541955] Lustre: Skipped 2 previous similar messages [ 5271.486480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.512538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.452553] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5273.456588] Lustre: Skipped 3 previous similar messages [ 5273.465162] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5273.474789] Lustre: Skipped 3 previous similar messages [ 5273.478747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.434985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.464082] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5275.466680] Lustre: Skipped 4 previous similar messages [ 5275.474125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.486546] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5275.500500] Lustre: Skipped 4 previous similar messages [ 5276.383833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.288932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.274482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.031582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.917310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.894779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.835739] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5281.841232] Lustre: Skipped 8 previous similar messages [ 5281.853356] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5281.866781] Lustre: Skipped 8 previous similar messages [ 5281.875096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.858518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.702241] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5283.711361] Lustre: Skipped 8 previous similar messages [ 5283.714751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.732807] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5283.743348] Lustre: Skipped 8 previous similar messages [ 5284.843782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.771618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.637100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.467838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.641665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.954990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.866904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.981592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.856460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.908405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.619328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.321876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.165170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.957170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.651481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.445804] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5298.453030] Lustre: Skipped 17 previous similar messages [ 5298.468866] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5298.475878] Lustre: Skipped 17 previous similar messages [ 5298.482787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.324663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.176738] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5300.179551] Lustre: Skipped 17 previous similar messages [ 5300.181540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.195676] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5300.201685] Lustre: Skipped 17 previous similar messages [ 5301.032592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.964639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.898145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.757503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.665715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.382303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.077585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.055528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.921824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.769980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.611381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.637221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.586480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.541380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.518260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.418697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.349533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.069355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.876917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.672868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.490759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.390196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.250954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.140040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.887081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.795963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.523653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.248611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.930537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.907926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.634553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.513697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.245720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.211916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.022185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.916488] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5330.926855] Lustre: Skipped 37 previous similar messages [ 5330.947853] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5330.955305] Lustre: Skipped 37 previous similar messages [ 5330.958786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.889280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.932771] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5332.935982] Lustre: Skipped 37 previous similar messages [ 5332.938092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.945447] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5332.947782] Lustre: Skipped 37 previous similar messages [ 5333.947884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.959412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.727159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.567327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.259975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.087508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.840636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.681463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.432618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.205572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.930392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.807480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.561305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.319318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.087122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.775622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.438987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.098994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.851489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.592983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.245315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.113562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.766211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.839465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.717229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.574827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.506562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.452519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.312742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.161314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.978688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.921474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.802136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.878347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.588576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.466724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.191543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.028274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.937669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.845649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.756929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.666717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.494529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.422502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.349961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.088395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.802215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.742167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.485613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.334836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.132585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.328221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.336904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.633703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.539465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.871284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.851536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.949946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.081702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.368233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.844654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.229329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.596415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.381782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.276744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.094634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.431618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.590705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.678326] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5395.683814] Lustre: Skipped 70 previous similar messages [ 5395.706994] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5395.719420] Lustre: Skipped 70 previous similar messages [ 5395.726427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.004934] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5397.008881] Lustre: Skipped 69 previous similar messages [ 5397.014115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.022221] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5397.026105] Lustre: Skipped 69 previous similar messages [ 5397.918537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.006903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.857469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.733763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.384444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.200751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.802596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.543757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.192812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.102431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.841703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.699451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.513929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.431232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.482995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.886557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.410689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.221452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.048464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.908155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.319948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.580588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.704882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.055230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.904174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.387647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.657180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.035279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.215421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.249903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.573390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.960545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.067572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.093697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.224615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.103631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.186041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.321579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.501570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.347447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.388612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.160336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.914643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.046310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.079739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.235838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.055621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.199420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.312693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.263603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.470806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.551825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.450521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.507714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.422905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.320434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.224331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.182097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.016423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.101122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.060860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.715589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.276949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.176029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.970949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.814146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.612742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.613054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.422624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.198510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.077532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.771034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.377570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.402546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.332876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.452768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.418379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.481362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.311274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.232975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.058658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.027749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.721846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.588055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.528361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.326226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.207558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.280789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.997946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.798039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.703241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.511769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.402475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.216283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.984339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.702108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.534446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.402715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.169181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.032438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.839726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.652837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.409630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.561414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.580942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.698398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.864521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.865309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.088271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.168195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.297056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.286109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.332225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.358807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.376566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.327885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.350105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.270382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.284210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.183936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.274650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.283994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.413551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.424413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.519260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.429326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.439837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.330659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.369831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.267769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.333506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.286036] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5524.299740] Lustre: Skipped 132 previous similar messages [ 5524.309767] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5524.321697] Lustre: Skipped 132 previous similar messages [ 5524.326591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.354528] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5525.368183] Lustre: Skipped 132 previous similar messages [ 5525.383807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.400579] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5525.403238] Lustre: Skipped 132 previous similar messages [ 5526.453107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.415203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.179042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.021380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.906902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.121747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.003168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.002669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.994011] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.051589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.891628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.896961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.856813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.868286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.854167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.771735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.670419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.708960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.714050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.515754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.298616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.235926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.028076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.950594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.770459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.734891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.795412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.669620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.588329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.428450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.336763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.377362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.192630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.902551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.717416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.553861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.375944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.193725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.901626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.596978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.580867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.715262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.690125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.670502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.630682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.490408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.366832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.062969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.893923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.739812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.120069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.146586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.172206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.043638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.250615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.349544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.175974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.053853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.701856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.456624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.284700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.938985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.591199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.501832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.583784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.681752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.993841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.805864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.770163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.613832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.405988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.390721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.528807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.411638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.175477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.045711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.075832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.000691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.798807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.512218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.451793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.212296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.923423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.850100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.786040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.648694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.330357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.136417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.975634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.638737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.588300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.557372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.500848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.206519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.300650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.418921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.619847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.723605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.623846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.524043] LustreError: 171193:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5616.538108] LustreError: 171193:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5616.552546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.683708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.063828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.310203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.771718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.588626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.770781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.820764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.123864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.384906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.300098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.194924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.290786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.359454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.317935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.282294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.128648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.177590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.094644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.198223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.316273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.270883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.290201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.154401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.154898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.954765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.115666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.047297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.618825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.249440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.029676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.831494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.865101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.053425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.158403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.109502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.055418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.802770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.695312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.651445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.586372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.316368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.147838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.172628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.430750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.232166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.196680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.119823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.170846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.250717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.535809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.719476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.696264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.704293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.565292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.619894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.567103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.595732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.367145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.154930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.993296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.200891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.161563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.426583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.351591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.258397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.349121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.466722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.348921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.607355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.695315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.538398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.526599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.271032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.219717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.225551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.077810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.735404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.429386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.233466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.947946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.834230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.558894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.515437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.241639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.297260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.450519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.574880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.672750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.570205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.678224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.565781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.569784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.638525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.536735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.433977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.237189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.006265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.564953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.386859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.111508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.870893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.681282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.493469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.211214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.117366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.800656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.547673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.508336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.393480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.327379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.265235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.931645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.990179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.682238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.491459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.343659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.126473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.249995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.269266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.188726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.127721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.325845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.633415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.725869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.565293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.564305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.692328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.828452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.655251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.688975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.771802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.769189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.944460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.963688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.878312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.841172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.805314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.573235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.493975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.262776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.189796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.185038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.105570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.073549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.144766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.022663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.894293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.819937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.791328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.661397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.683958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.576703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.631672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.715991] LustreError: 171193:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5763.734357] LustreError: 171193:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5763.749318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.565299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.565526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.578397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.462731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.304339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.403794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.294599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.350952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.243651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.225083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.197503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.399817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.409310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.424708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.271819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.198032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.023954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.828508] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5780.832647] Lustre: Skipped 272 previous similar messages [ 5780.847144] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5780.853135] Lustre: Skipped 272 previous similar messages [ 5780.856054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.597798] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5781.602664] Lustre: Skipped 272 previous similar messages [ 5781.606784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.615151] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5781.621202] Lustre: Skipped 272 previous similar messages [ 5782.398221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.223867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.366307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.360193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.274653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.183835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.238403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.193382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.292642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.018581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.860569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.687909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.550937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.471359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.545033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.331234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.278265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.141712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.942678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.937659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.735242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.718521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.597110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.643169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.327878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.091831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.789305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.652521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.598434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.625348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.566902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.702494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.813428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.939523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.942499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.845556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.798275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.846318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.801506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.734709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.491270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.421321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.331342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.500460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.324843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.261870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.224184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.308515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.457415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.363688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.419735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.417339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.517166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.469599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.490902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.552735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.497469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.524662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.476265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.211364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.139441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.965025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.043311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.954532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.890779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.754898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.707683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.574743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.464626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.344758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.960892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.716679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.580210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.394272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.282459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.385568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.091568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.293511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.298027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.218083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.266752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.402336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.101057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.977535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.789160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.687099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.321719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.567726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.633812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.562972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.397859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.030995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.006238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.711612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.368529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.957960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.985930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.824728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.721072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.582322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.494922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.283176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.211295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.064635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.155163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.215684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.104708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.852529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.768362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.710259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.462582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.294719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.815744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.484290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.444300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.174269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.895688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.922978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.591963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.213092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.838205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.582734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.272722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.894256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.830679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.530522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.200503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.426800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.372063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.562363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.591822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.798954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.755558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.646765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.915336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.999752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.158742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.858464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.998511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.104152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.911428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.680396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.680021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.845653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.200780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.105589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.080304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.156166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.910906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.775302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.893512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.614733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.465628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.301059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.456298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.604512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.285573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.453415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.554708] LustreError: 171193:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5927.560931] LustreError: 171193:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5927.578577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.579436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.222062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.946651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.752626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.401378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.117385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.689843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.544504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.425280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.186841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.921589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.706354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.346547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.087228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.826679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.519818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.316707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.594804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.317306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.010824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.651503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.886403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.159931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.110739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.962563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.729533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.757748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.645631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.737967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.877191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.835490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.814862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.780353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.757389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.615420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.665915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.366913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.216065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.133703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.009773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.709724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.542343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.399115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.428395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.342084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.238056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.205474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.214995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.048931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.032135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.058396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.976480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.843892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.819339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.509053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.515347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.457650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.571828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.710286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.713126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.917740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.891793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.033562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.155496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.985262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.131336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.308858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.292282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.370819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.492046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.466990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.537902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.513820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.590438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.496233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.436540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.396375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.281446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.293906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.409861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.545201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.467060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.339084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.099742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.989592] LustreError: 171193:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6005.993639] LustreError: 171193:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6006.001567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.584331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.535922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.345319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.215267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.052710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.135878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.216823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.179895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.234598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.475368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.435034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.594331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.480539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.393478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.201531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.211215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.136977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.204974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.216263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.223917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.263457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.406144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.690923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.078506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.687932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.208795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.594559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.925902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.861360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.991871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.249676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.375839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.617631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.791938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.999874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.055573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.133439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.011192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.608207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.393881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.382577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.319943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.557384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.374136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.415922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.127541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.998881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.846790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.718581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.628788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.400803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.194462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.911649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.610963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.281236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.269518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.211561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.049200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.000360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.039633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.805457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.833762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.628197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.587493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.492281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.418963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.218334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.082051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.853650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.760966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.470800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.495809] LustreError: 171193:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6076.507700] LustreError: 171193:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6076.531577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.242377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.337772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.145854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.374874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.560909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.462234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.748973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.820176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.085583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.174527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.131684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.672659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.907676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.046188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.011188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.396857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.554290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.542283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.564721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.308969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.165953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.007049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.853381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.586305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.258580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.810899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.643511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.473140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.224295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.974506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.700374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.315606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.166445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.974560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.036506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.894734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.898710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.672870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.316181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.896497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.702863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.610925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.645256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.505425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.541113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.558528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.641901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.705086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.397769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.097392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.005959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.107466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.310398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.261321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.038601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.979821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.831111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.814624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.912813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.635183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.649058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.660811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.710933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.757422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.760970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.868272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.728493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.694934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.821644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.897592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.932040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.015861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.131409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.962269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.996293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.771589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.630874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.481470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.404317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.219575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.179707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.096777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.113270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.019635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.916083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.855777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.777681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.683648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.593214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.356909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.241614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.163175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.991398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.007561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.032317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.851429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.755830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.553416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.308579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.348805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.324729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.321722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.287264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.097153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.990368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.967835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.910236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.819305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.659323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.665917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.662736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.633461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.476137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.326295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.159880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.000420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.917917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.899371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.808871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.682551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.555082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.671539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.721137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.742475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.891318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.806826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.453294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.410080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.386608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.204517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.031754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.707832] LustreError: 172996:0:(ofd_dev.c:1782:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6506.724592] LustreError: 171193:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6932.460739] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6932.483087] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6932.519013] Lustre: Skipped 448 previous similar messages [ 6932.544075] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6936.034384] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6936.057975] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6937.890466] Lustre: server umount lustre-OST0000 complete [ 6943.717452] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6943.741713] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6943.750322] Lustre: Skipped 1 previous similar message [ 6946.785241] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6946.809139] Lustre: Skipped 1 previous similar message [ 6949.039753] Lustre: server umount lustre-MDT0000 complete [ 6951.907060] LustreError: 196409:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6951.940835] LustreError: 196409:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6953.432398] LustreError: 171154:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781279630 with bad export cookie 10763477252409025730 [ 6953.434983] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6953.446838] LustreError: 171154:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6953.885991] Lustre: server umount lustre-MDT0001 complete [ 6965.278796] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 6968.303713] Key type lgssc unregistered [ 6968.584595] LNet: 226372:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6968.595966] LNetError: 226372:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6968.612290] LNet: Removed LNI 192.168.201.135@tcp [ 6969.712234] Key type .llcrypt unregistered [ 6969.718633] Key type ._llcrypt unregistered [ 6994.791436] Key type ._llcrypt registered [ 6994.793224] Key type .llcrypt registered [ 6994.917766] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 6995.961337] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6996.076911] alg: No test for adler32 (adler32-zlib) [ 6997.252986] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [ 6997.480765] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 6999.144136] Key type lgssc registered [ 7000.511217] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7008.053544] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7008.078188] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7009.564028] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7014.156747] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7019.005555] LustreError: 227789:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7019.019508] LustreError: 227789:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7023.504630] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7024.181397] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7029.024794] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7037.131129] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7042.474908] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7050.233178] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7050.754176] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7054.832048] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7057.010917] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7065.079949] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7074.099660] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7075.785409] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7075.796063] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7075.799017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7075.806666] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7075.811199] LustreError: 227808:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7078.246776] Lustre: lustre-OST0000: Not available for connect from 192.168.201.35@tcp (stopping) [ 7079.400872] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7079.412563] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7079.430678] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7080.425117] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7080.457154] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7080.459871] Lustre: Skipped 1 previous similar message [ 7083.337497] Lustre: lustre-OST0000: Not available for connect from 192.168.201.35@tcp (stopping) [ 7084.362774] Lustre: server umount lustre-OST0000 complete [ 7095.895126] LustreError: 227808:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7118.568791] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7118.888046] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7120.951113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7120.978042] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7124.957697] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7130.211354] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7130.239407] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7132.180226] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7135.385286] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 1 sec [ 7141.395758] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff92c545e43800.ost_server_uuid 50 [ 7142.866694] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff92c545e43800.ost_server_uuid in FULL state after 0 sec [ 7148.000429] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7148.009504] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7148.021205] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7151.073226] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7152.923583] Lustre: server umount lustre-OST0000 complete [ 7157.218565] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7157.225702] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7157.240087] Lustre: Skipped 1 previous similar message [ 7157.255356] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7157.257986] Lustre: Skipped 2 previous similar messages [ 7163.297910] Lustre: server umount lustre-MDT0000 complete [ 7166.436941] LustreError: 229630:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7166.454755] LustreError: 229630:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7167.516988] LustreError: 227770:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781279844 with bad export cookie 14618830712944432480 [ 7167.538153] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7167.902780] Lustre: server umount lustre-MDT0001 complete [ 7180.818277] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 7183.875409] Key type lgssc unregistered [ 7184.213707] LNet: 232992:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7184.224909] LNetError: 232992:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7185.254096] LNet: Removed LNI 192.168.201.135@tcp [ 7186.248707] Key type .llcrypt unregistered [ 7186.250939] Key type ._llcrypt unregistered [ 7214.446647] Key type ._llcrypt registered [ 7214.449630] Key type .llcrypt registered [ 7214.552551] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 7215.762447] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7215.773182] alg: No test for adler32 (adler32-zlib) [ 7216.806388] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [ 7217.000747] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 7218.655430] Key type lgssc registered [ 7219.620830] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7227.985881] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7228.004331] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7229.591111] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7233.584889] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7239.143058] LustreError: 234702:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7239.166947] LustreError: 234702:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7241.150374] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7241.482482] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7245.668203] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7248.681393] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7255.646621] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7256.302406] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7262.654553] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7273.036381] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7273.257352] Lustre: lustre-OST0001: new disk, initializing [ 7273.260667] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7273.266960] Lustre: 237011:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7273.393239] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7274.412646] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7278.316528] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7281.267925] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7281.281350] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7281.447951] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7289.467858] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7297.004167] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7309.799429] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [ 7309.812379] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7309.836287] LustreError: 234703:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7311.321552] Lustre: server umount lustre-MDT0000 complete [ 7312.354830] LustreError: 234702:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7312.357552] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7312.378188] Lustre: Skipped 2 previous similar messages [ 7314.731612] LustreError: 234682:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781279991 with bad export cookie 8397661751075211108 [ 7314.735011] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7314.743276] LustreError: 235439:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 7314.952492] Lustre: server umount lustre-MDT0001 complete [ 7328.516829] Lustre: server umount lustre-OST0000 complete [ 7341.213313] Lustre: server umount lustre-OST0001 complete [ 7360.142533] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 7370.969945] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7371.594823] LustreError: 240120:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7371.623014] LustreError: 240120:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7371.735984] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7376.126923] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7376.876179] LustreError: 240119:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7382.169880] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7382.504565] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7385.662669] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7387.770961] Lustre: 241272:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7392.357186] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7393.641732] LustreError: 241625:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7393.649505] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 7393.654208] LustreError: 241625:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7396.724438] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7397.990836] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7408.011584] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7408.349680] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7408.365907] Lustre: Skipped 1 previous similar message [ 7415.370477] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7423.035752] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7425.914324] Lustre: 243145:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7434.213394] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7434.220811] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7434.228446] Lustre: Skipped 1 previous similar message [ 7439.333585] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7439.337823] Lustre: Skipped 1 previous similar message [ 7440.149949] Lustre: server umount mylustre-MDT0000 complete [ 7443.321564] LustreError: 240095:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781280120 with bad export cookie 8397661751075213957 [ 7443.326982] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7443.338712] LustreError: 240095:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7443.732498] Lustre: server umount mylustre-MDT0001 complete [ 7457.462382] Lustre: server umount mylustre-OST0000 complete [ 7460.642473] Lustre: 233965:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781280121/real 1781280121] req@ffff9bf059aa8700 x1867807346284672/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781280137 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7460.679948] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7460.703203] Lustre: Skipped 3 previous similar messages [ 7461.787406] Lustre: server umount mylustre-OST0001 complete [ 7480.074886] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 7489.077606] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7489.479132] LustreError: 245453:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7489.485742] LustreError: 245453:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 7489.546036] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7493.033099] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7499.365295] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7503.243444] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7505.349180] Lustre: 246593:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7509.969819] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7515.519755] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7516.017766] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7523.367364] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7524.615023] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 7529.264045] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7537.806261] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7541.434403] Lustre: 248462:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7550.439662] Lustre: tfs-MDT0000-lwp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7550.441135] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7550.446215] Lustre: Skipped 1 previous similar message [ 7550.458538] Lustre: Skipped 6 previous similar messages [ 7555.276026] Lustre: server umount tfs-MDT0000 complete [ 7555.573718] LustreError: 245454:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7555.598543] LustreError: 245454:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 7558.933208] LustreError: 245429:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781280235 with bad export cookie 8397661751075215910 [ 7558.940102] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7558.950876] LustreError: 245429:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 7559.273663] Lustre: server umount tfs-MDT0001 complete [ 7573.594808] Lustre: server umount tfs-OST0000 complete [ 7576.547589] Lustre: 233964:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781280237/real 1781280237] req@ffff9bf17e595500 x1867807346333952/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781280253 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7576.599307] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7576.620182] Lustre: Skipped 2 previous similar messages [ 7578.561597] Lustre: server umount tfs-OST0001 complete [ 7602.353244] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 7610.123271] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7610.575840] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7610.581503] Lustre: Skipped 3 previous similar messages [ 7614.838301] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7620.069726] LustreError: 250713:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7620.092899] LustreError: 250713:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7622.043064] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7626.552794] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7630.109544] Lustre: 251858:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7637.643290] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7645.295551] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7647.188257] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 7653.437505] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7655.668661] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 7659.951338] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7668.550720] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7672.947551] Lustre: 253728:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7680.571255] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 12:05:55 (1781280355) [ 7696.867052] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7696.874594] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7696.884385] Lustre: Skipped 1 previous similar message [ 7696.900943] Lustre: Skipped 3 previous similar messages [ 7701.988677] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7701.998705] Lustre: Skipped 2 previous similar messages [ 7703.260043] Lustre: server umount lustre-MDT0000 complete [ 7707.178676] LustreError: 251457:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781280384 with bad export cookie 8397661751075217604 [ 7707.187638] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7707.196383] LustreError: 251457:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7712.230227] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7712.237128] Lustre: Skipped 1 previous similar message [ 7713.890450] Lustre: server umount lustre-MDT0001 complete [ 7732.191908] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7732.896265] Lustre: server umount lustre-OST0000 complete [ 7744.104438] Lustre: server umount lustre-OST0001 complete [ 7754.684700] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 7764.742593] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 7808.727722] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7808.727722] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7808.727722] [ 7808.745610] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7808.984580] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7809.036412] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7809.121117] Lustre: lustre-MDT0000: new disk, initializing [ 7809.229035] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7809.235194] Lustre: Skipped 3 previous similar messages [ 7809.249192] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7814.454783] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7825.868664] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 7825.868664] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7825.868664] [ 7825.902844] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7826.054148] Lustre: 256915:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7826.128614] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7826.131481] Lustre: Skipped 1 previous similar message [ 7826.194083] Lustre: lustre-MDT0001: new disk, initializing [ 7826.353790] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7826.376121] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7831.017493] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7842.462280] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7842.909764] Lustre: lustre-OST0000: new disk, initializing [ 7842.915673] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7842.924814] Lustre: 258498:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7844.968530] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7844.975435] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7845.118441] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7851.662729] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7862.676804] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7862.861853] Lustre: lustre-OST0001: new disk, initializing [ 7862.864886] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7862.870810] Lustre: 259369:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7864.316196] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7864.325749] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7864.387744] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7869.583817] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 7881.247741] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 12:09:16 (1781280556) [ 7895.453363] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 12:09:30 (1781280570) [ 7915.503220] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7915.517676] Lustre: Skipped 4 previous similar messages [ 7915.521820] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7915.532621] Lustre: Skipped 1 previous similar message [ 7919.789757] Lustre: server umount lustre-OST0000 complete [ 7920.614432] LustreError: 258482:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7920.651369] LustreError: 258482:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 9 previous similar messages [ 7930.259519] Lustre: server umount lustre-MDT0000 complete [ 7934.428088] LustreError: 256906:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781280611 with bad export cookie 8397661751075219158 [ 7934.440329] LustreError: 256906:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7934.440727] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7940.740380] Lustre: server umount lustre-MDT0001 complete [ 7953.013212] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 7965.053893] Lustre: server umount lustre-OST0001 complete [ 7974.701977] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 7992.112735] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 8032.846224] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8033.114983] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8033.152936] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8033.248905] Lustre: lustre-MDT0000: new disk, initializing [ 8033.370851] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8037.584777] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8050.027369] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8050.227588] Lustre: 263938:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8050.358949] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8050.365115] Lustre: Skipped 1 previous similar message [ 8050.474555] Lustre: lustre-MDT0001: new disk, initializing [ 8050.719504] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8050.751996] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8056.553247] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8068.543472] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8075.075729] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8083.095401] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8083.554588] Lustre: lustre-OST0000: new disk, initializing [ 8083.564457] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8083.583103] Lustre: 265906:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8083.777267] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8083.782759] Lustre: Skipped 5 previous similar messages [ 8084.844885] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8084.852040] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8085.090518] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8093.480284] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8104.553295] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8110.562212] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8110.570672] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8110.579699] Lustre: Skipped 5 previous similar messages [ 8110.584739] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8110.590545] Lustre: Skipped 5 previous similar messages [ 8115.685604] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8116.780405] Lustre: server umount lustre-OST0000 complete [ 8126.812553] Lustre: server umount lustre-MDT0000 complete [ 8131.076881] LustreError: 263930:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781280808 with bad export cookie 8397661751075220621 [ 8131.090501] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8131.660634] Lustre: server umount lustre-MDT0001 complete [ 8142.047878] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 8144.485438] Key type lgssc unregistered [ 8144.836500] LNet: 267666:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8144.841192] LNetError: 267666:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8144.851868] LNet: Removed LNI 192.168.201.135@tcp [ 8145.619319] Key type .llcrypt unregistered [ 8145.621667] Key type ._llcrypt unregistered [ 8178.567285] Key type ._llcrypt registered [ 8178.569509] Key type .llcrypt registered [ 8178.820077] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 8180.697366] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8180.737839] alg: No test for adler32 (adler32-zlib) [ 8182.475588] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [ 8183.064098] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 8185.175204] Key type lgssc registered [ 8186.217291] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8194.679973] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8194.712132] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8196.360583] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8200.390613] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8206.826449] LustreError: 269132:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8206.853070] LustreError: 269132:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8208.257286] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8208.833875] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8213.433691] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8221.393419] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8227.355398] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8235.272267] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8235.805448] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8242.004810] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8244.794122] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8252.921559] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8254.550670] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8258.530025] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8258.538427] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8258.551961] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8258.586291] Lustre: Skipped 1 previous similar message [ 8260.420422] Lustre: server umount lustre-OST0000 complete [ 8268.772077] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8268.785891] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8268.801355] Lustre: Skipped 1 previous similar message [ 8268.808620] Lustre: Skipped 1 previous similar message [ 8272.035386] Lustre: server umount lustre-MDT0000 complete [ 8273.895235] LustreError: 269860:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8273.916551] LustreError: 269860:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8276.299146] LustreError: 269113:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781280953 with bad export cookie 5146793448990408947 [ 8276.308123] LustreError: 269113:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8276.311335] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8276.845106] Lustre: server umount lustre-MDT0001 complete [ 8287.916463] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 8290.861294] Key type lgssc unregistered [ 8291.150777] LNet: 272717:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8291.163398] LNetError: 272717:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8291.184992] LNet: Removed LNI 192.168.201.135@tcp [ 8292.083517] Key type .llcrypt unregistered [ 8292.088722] Key type ._llcrypt unregistered [ 8309.482941] Key type ._llcrypt registered [ 8309.485274] Key type .llcrypt registered [ 8309.637789] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 8310.622322] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8310.699585] alg: No test for adler32 (adler32-zlib) [ 8311.786457] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [ 8312.049798] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 8313.671167] Key type lgssc registered [ 8314.954774] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8324.248205] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8324.262778] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8339.808065] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8344.927262] LustreError: 273910:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.135@tcp: failed processing log, type 4: rc = -110 [ 8371.679176] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8379.725724] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8389.373534] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8391.996579] Lustre: server umount lustre-OST0000 complete [ 8408.939785] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 8411.189743] Key type lgssc unregistered [ 8411.468442] LNet: 275545:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8411.484657] LNetError: 275545:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8411.512385] LNet: Removed LNI 192.168.201.135@tcp [ 8412.122161] Key type .llcrypt unregistered [ 8412.123126] Key type ._llcrypt unregistered [ 8452.038621] Key type ._llcrypt registered [ 8452.040577] Key type .llcrypt registered [ 8452.193039] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 8453.280447] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8453.438145] alg: No test for adler32 (adler32-zlib) [ 8454.491336] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [ 8454.753524] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 8456.471199] Key type lgssc registered [ 8457.608749] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8472.185564] loop0: detected capacity change from 0 to 204800000 [ 8472.443501] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8472.459863] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8472.867443] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8474.161606] loop0: detected capacity change from 0 to 204800000 [ 8474.389091] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8474.397713] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8475.035639] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8476.231659] loop0: detected capacity change from 0 to 204800000 [ 8476.471591] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8476.481035] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8476.616746] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8477.633197] loop0: detected capacity change from 0 to 204800000 [ 8477.809743] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8477.839591] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8478.040786] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8479.282433] loop0: detected capacity change from 0 to 204800000 [ 8479.311094] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8483.873622] loop0: detected capacity change from 0 to 204800000 [ 8483.907725] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8488.441589] loop0: detected capacity change from 0 to 204800000 [ 8488.497086] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8492.919341] loop0: detected capacity change from 0 to 204800000 [ 8492.943440] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8497.486852] loop0: detected capacity change from 0 to 204800000 [ 8497.559511] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8497.618430] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8497.729855] Lustre: 278960:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8497.735807] Lustre: 278960:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8497.743825] Lustre: 278960:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8497.753434] Lustre: 278960:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8497.763784] Lustre: 278960:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8497.780940] Lustre: 278960:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8497.795147] Lustre: 278960:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8497.805862] Lustre: 278960:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8497.816851] Lustre: 278960:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8497.835744] Lustre: 278960:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8497.842247] Lustre: 278960:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8497.859219] Lustre: 278960:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8497.871468] Lustre: 278960:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8497.892163] Lustre: 278960:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8503.805992] Lustre: server umount MGS complete [ 8504.792237] loop0: detected capacity change from 0 to 204800000 [ 8504.889097] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8505.557300] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8505.592975] Lustre: lustre-MDT0000: Aborting client recovery [ 8506.660418] loop1: detected capacity change from 0 to 204800000 [ 8506.799936] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8506.969234] Lustre: 279392:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8506.976730] Lustre: 279392:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8506.985311] Lustre: 279392:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8506.994985] Lustre: 279392:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8507.004760] Lustre: 279392:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8507.015389] Lustre: 279392:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8507.033084] Lustre: 279392:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8507.071041] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8507.370343] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8507.413976] Lustre: lustre-MDT0001: Aborting client recovery [ 8508.464833] loop2: detected capacity change from 0 to 204800000 [ 8508.602141] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8508.734787] Lustre: 279471:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8508.757276] Lustre: 279471:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8508.774137] Lustre: 279471:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8508.784741] Lustre: 279471:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8508.797451] Lustre: 279471:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8508.805025] Lustre: 279471:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8508.821910] Lustre: 279471:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8508.921554] Lustre: Found index 0 for lustre-OST0000, updating log [ 8508.963764] Lustre: 279490:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8509.078989] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8509.089608] LustreError: 279336:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 4, retries 0, failed: rc = -108 [ 8509.118925] Lustre: lustre-OST0000: Aborting recovery [ 8510.261869] loop3: detected capacity change from 0 to 204800000 [ 8510.500529] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8510.664459] Lustre: 279555:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8510.679149] Lustre: 279555:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8510.692214] Lustre: 279555:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8510.708077] Lustre: 279555:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8510.730178] Lustre: 279555:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8510.742633] Lustre: 279555:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8510.758355] Lustre: 279555:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8510.788292] Lustre: Found index 1 for lustre-OST0001, updating log [ 8510.888527] Lustre: 279566:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8511.064268] Lustre: lustre-OST0001: Aborting recovery [ 8515.128740] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8515.132815] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8515.170169] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8515.195171] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8517.167314] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8517.174714] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8520.489698] LustreError: 279303:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 704us: rc = -2 [ 8521.016422] LustreError: 279303:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 552us: rc = -2 [ 8521.041185] LustreError: 279303:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 17 previous similar messages [ 8537.568250] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8537.571873] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8537.601062] Lustre: Skipped 2 previous similar messages [ 8537.611136] Lustre: Skipped 1 previous similar message [ 8542.693805] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8542.698100] Lustre: Skipped 2 previous similar messages [ 8547.819820] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8547.835410] Lustre: Skipped 3 previous similar messages [ 8548.319170] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8548.892091] Lustre: server umount lustre-MDT0000 complete [ 8549.735515] LustreError: 279286:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781281226 with bad export cookie 4985008806270591714 [ 8549.742909] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8549.749870] LustreError: 279286:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8552.931436] LustreError: 279300:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8552.934650] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8552.936073] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8552.949643] LustreError: 279300:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8552.981957] Lustre: Skipped 2 previous similar messages [ 8554.975801] LustreError: 279302:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8554.999345] LustreError: 279302:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8559.072511] LustreError: 279302:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8559.072568] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8559.096234] LustreError: 279302:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8559.117327] Lustre: Skipped 4 previous similar messages [ 8564.191142] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8564.196572] LustreError: 279962:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8564.215910] LustreError: 279962:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8564.460344] Lustre: server umount lustre-MDT0001 complete [ 8571.797342] Lustre: server umount lustre-OST0000 complete [ 8587.231685] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8587.455775] Lustre: server umount lustre-OST0001 complete [ 8595.059498] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 12:21:10 (1781281270) [ 8624.589107] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 8635.846660] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 8674.645794] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8674.913345] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8674.940334] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8675.013373] Lustre: lustre-MDT0000: new disk, initializing [ 8675.139615] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8675.153294] Lustre: Skipped 1 previous similar message [ 8675.171240] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8680.027877] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8691.078302] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8691.213915] Lustre: 283641:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8691.269209] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8691.272366] Lustre: Skipped 1 previous similar message [ 8691.331328] Lustre: lustre-MDT0001: new disk, initializing [ 8691.487329] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8691.537753] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8691.547303] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8696.226293] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8705.934813] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8711.214957] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8717.247780] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8717.444510] Lustre: lustre-OST0000: new disk, initializing [ 8717.447477] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8717.450815] Lustre: 285608:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8717.525231] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8723.466543] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8727.633642] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8727.646953] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8727.756682] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8733.473816] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8739.546397] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8754.092196] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8760.334825] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8820.198340] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8820.201019] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8820.208183] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8820.208190] Lustre: Skipped 2 previous similar messages [ 8820.233559] Lustre: Skipped 1 previous similar message [ 8821.589116] Lustre: server umount lustre-OST0000 complete [ 8825.318673] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8831.453770] Lustre: server umount lustre-MDT0000 complete [ 8835.579847] LustreError: 283650:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8835.621584] LustreError: 283650:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8836.228455] LustreError: 283635:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781281513 with bad export cookie 4985008806270594528 [ 8836.232368] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8836.234092] LustreError: 283635:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8836.614429] Lustre: server umount lustre-MDT0001 complete [ 8845.566671] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8849.518890] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8854.667794] Lustre: server umount MGS complete [ 8868.042883] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8868.433972] LustreError: 289524:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8868.485737] LustreError: 289537:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8868.546437] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8873.229794] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8882.717945] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8887.047477] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8894.153861] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8899.443776] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8906.082584] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8906.313211] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8906.320977] Lustre: Skipped 1 previous similar message [ 8913.762130] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 8922.362286] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8954.853841] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8954.857278] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8954.866315] Lustre: Skipped 1 previous similar message [ 8954.872831] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8954.873234] LustreError: Skipped 2 previous similar messages [ 8954.902143] Lustre: Skipped 6 previous similar messages [ 8966.113407] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8966.501876] Lustre: server umount lustre-OST0000 complete [ 8975.337384] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8975.343533] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8975.357784] Lustre: Skipped 1 previous similar message [ 8977.884280] Lustre: server umount lustre-MDT0000 complete [ 8980.453249] LustreError: 291368:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8980.467287] LustreError: 291368:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8981.690545] LustreError: 289506:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781281658 with bad export cookie 4985008806270596369 [ 8981.699776] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8981.706714] LustreError: 289506:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8982.316570] Lustre: server umount lustre-MDT0001 complete [ 8993.497315] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 8997.082697] Key type lgssc unregistered [ 8997.493095] LNet: 293472:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8997.503734] LNetError: 293472:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8997.520732] LNet: Removed LNI 192.168.201.135@tcp [ 8998.288674] Key type .llcrypt unregistered [ 8998.290877] Key type ._llcrypt unregistered [ 9034.358214] Key type ._llcrypt registered [ 9034.361257] Key type .llcrypt registered [ 9034.491690] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 9051.521813] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 9052.653367] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9052.885260] alg: No test for adler32 (adler32-zlib) [ 9054.033856] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [ 9054.280435] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 9055.992510] Key type lgssc registered [ 9057.505840] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9095.808298] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9095.869877] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9097.213106] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9097.247521] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9097.315341] Lustre: lustre-MDT0000: new disk, initializing [ 9097.392298] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9097.418319] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9102.964338] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 9115.930852] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9116.153539] Lustre: 297434:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9116.232388] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9116.238820] Lustre: Skipped 1 previous similar message [ 9116.358485] Lustre: lustre-MDT0001: new disk, initializing [ 9116.495771] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9116.518716] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9116.530036] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9121.962529] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 9132.124947] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9137.298477] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9144.085692] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9144.254937] Lustre: lustre-OST0000: new disk, initializing [ 9144.260195] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9144.268953] Lustre: 299406:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9144.345897] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9152.370419] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 9152.687193] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9152.705729] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9152.852794] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9161.798533] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9169.155422] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9180.566255] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9189.073645] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9198.617082] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9240.040243] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9240.045071] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9240.049660] LustreError: Skipped 1 previous similar message [ 9240.061125] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9240.070103] Lustre: Skipped 1 previous similar message [ 9240.088075] Lustre: Skipped 1 previous similar message [ 9245.157290] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9245.167597] Lustre: Skipped 1 previous similar message [ 9250.277801] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9250.284725] Lustre: Skipped 1 previous similar message [ 9252.319143] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9252.484535] Lustre: server umount lustre-OST0000 complete [ 9260.001508] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9260.008787] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9260.028898] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9263.170481] Lustre: server umount lustre-MDT0000 complete [ 9267.290351] LustreError: 297427:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781281944 with bad export cookie 6599779019138394560 [ 9267.296986] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9267.299860] LustreError: 297427:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9267.944451] Lustre: server umount lustre-MDT0001 complete [ 9276.941357] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9282.501742] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 9287.384296] Lustre: server umount MGS complete [ 9299.346630] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9299.782091] LustreError: 303324:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9299.811086] LustreError: 303337:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9299.812163] LustreError: 303324:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9299.897972] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9305.074846] LustreError: 303325:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9305.388589] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 9309.165656] LustreError: 303324:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9314.278954] LustreError: 303325:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9314.849736] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9315.528957] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9321.234312] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 9329.122763] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9334.898631] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9342.420168] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9342.847225] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9349.918777] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 9357.412713] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9385.958556] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9385.959614] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9385.969695] LustreError: Skipped 1 previous similar message [ 9385.970744] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9385.970747] Lustre: Skipped 2 previous similar messages [ 9385.997560] Lustre: Skipped 2 previous similar messages [ 9387.583959] Lustre: server umount lustre-OST0000 complete [ 9392.610891] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9392.618058] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9394.154586] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9394.170174] Lustre: Skipped 2 previous similar messages [ 9397.672687] Lustre: server umount lustre-MDT0000 complete [ 9399.264722] LustreError: 303325:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9399.291570] LustreError: 303325:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9400.892756] LustreError: 303305:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781282077 with bad export cookie 6599779019138396485 [ 9400.897406] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9400.899420] LustreError: 303305:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9401.217741] Lustre: server umount lustre-MDT0001 complete [ 9411.077899] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [ 9413.591256] Key type lgssc unregistered [ 9413.835722] LNet: 307271:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9413.841631] LNetError: 307271:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9414.890711] LNet: Removed LNI 192.168.201.135@tcp [ 9415.602874] Key type .llcrypt unregistered [ 9415.605578] Key type ._llcrypt unregistered [ 9439.731655] Key type ._llcrypt registered [ 9439.734758] Key type .llcrypt registered [ 9440.001495] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [ 9453.323550] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 9454.703824] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9454.769945] alg: No test for adler32 (adler32-zlib) [ 9455.883405] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [ 9456.164661] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [ 9457.895216] Key type lgssc registered [ 9459.206109] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9489.113462] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [ 9506.625573] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9506.652067] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9508.036477] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9508.163201] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9508.275660] Lustre: lustre-MDT0000: new disk, initializing [ 9508.410327] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9508.458563] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9514.084301] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 9525.549617] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9525.894151] Lustre: lustre-OST0000: new disk, initializing [ 9525.903428] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9525.914271] Lustre: Skipped 1 previous similar message [ 9525.926586] Lustre: 311304:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9525.996322] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9532.417846] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [ 9534.546373] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9534.564429] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9534.590953] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10305.505054] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10305.517464] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10307.732680] Lustre: server umount lustre-MDT0000 complete [10311.742863] LustreError: 310491:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781282988 with bad export cookie 12753744903656814689 [10311.750653] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10311.840212] Lustre: server umount lustre-OST0000 complete [10336.159722] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [10347.080972] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [10383.045938] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10383.408637] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10383.467799] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10383.562484] Lustre: lustre-MDT0000: new disk, initializing [10383.663551] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10383.686213] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10390.062720] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [10402.910943] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10403.109542] Lustre: 315639:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10403.222759] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10403.225346] Lustre: Skipped 1 previous similar message [10403.340878] Lustre: lustre-MDT0001: new disk, initializing [10403.519587] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10403.579916] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10403.590091] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10408.835729] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [10419.234412] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10425.633736] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10432.606112] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10432.815274] Lustre: lustre-OST0000: new disk, initializing [10432.823848] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10432.838591] Lustre: 317606:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10432.989972] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10439.014298] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [10442.853667] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10442.870493] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10443.040105] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10450.227782] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10455.339781] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10455.558246] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10460.401341] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10460.681194] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10463.711915] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10463.713942] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10463.715580] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10463.725343] Lustre: Skipped 1 previous similar message [10463.735609] Lustre: Skipped 1 previous similar message [10468.839983] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10468.845664] Lustre: Skipped 1 previous similar message [10469.104405] Lustre: server umount lustre-OST0000 complete [10473.952732] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10473.959635] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10473.973579] Lustre: Skipped 1 previous similar message [10473.989450] Lustre: Skipped 1 previous similar message [10479.078910] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10479.085951] Lustre: Skipped 1 previous similar message [10487.263194] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10487.672505] Lustre: server umount lustre-MDT0000 complete [10491.919391] LustreError: 315632:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781283168 with bad export cookie 12753744903659755508 [10491.940912] LustreError: 315632:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10491.943134] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10492.326632] Lustre: server umount lustre-MDT0001 complete [10496.240576] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10498.181586] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 12:52:53 (1781283173) [10506.129140] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10506.818914] LustreError: 319920:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10506.837785] LustreError: 319920:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10506.948684] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10511.730528] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [10512.353873] LustreError: 319921:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10516.451711] LustreError: 319920:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10521.359461] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10521.573121] LustreError: 319921:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10521.843507] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10527.547119] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [10534.967950] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10541.167169] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10548.941709] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10549.302650] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10556.337575] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [10564.640881] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10571.634654] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10571.840630] Lustre: lustre-OST0001: new disk, initializing [10571.844417] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10571.851200] Lustre: 322755:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10571.932401] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10578.528844] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10578.548069] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10578.616048] LustreError: 323073:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10578.930485] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [10579.687531] LustreError: 323073:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10579.701515] LustreError: 323073:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10580.707299] LustreError: 323081:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10582.754073] LustreError: 323081:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10582.767230] LustreError: 323081:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10586.917884] LustreError: 323081:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10586.928153] LustreError: 323081:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [10588.667027] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10588.845255] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10593.836205] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff92c558390800.ost_server_uuid 50 [10595.040282] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff92c558390800.ost_server_uuid in FULL state after 0 sec [10595.109308] LustreError: 323081:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10595.114622] LustreError: 323081:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10600.186916] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff92c558390800.ost_server_uuid 50 [10601.577227] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff92c558390800.ost_server_uuid in FULL state after 0 sec [10603.322575] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10618.856269] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10618.860361] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10618.862016] LustreError: Skipped 2 previous similar messages [10618.872969] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10618.889028] Lustre: Skipped 1 previous similar message [10618.896499] Lustre: Skipped 2 previous similar messages [10622.125541] Lustre: server umount lustre-OST0001 complete [10624.849330] LustreError: 321742:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.201.35@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10630.119122] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10634.430236] Lustre: server umount lustre-OST0000 complete [10639.841052] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10639.849464] LustreError: Skipped 1 previous similar message [10639.856253] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10639.881194] Lustre: Skipped 1 previous similar message [10639.884498] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10639.888307] Lustre: Skipped 6 previous similar messages [10645.476185] LustreError: 320865:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10645.478835] Lustre: server umount lustre-MDT0000 complete [10645.512466] LustreError: 320865:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [10649.269688] LustreError: 324289:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781283326 with bad export cookie 12753744903659756390 [10649.272695] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10649.279047] LustreError: 324289:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10649.596942] Lustre: server umount lustre-MDT0001 complete [10661.245474] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [10664.837754] Key type lgssc unregistered [10665.196821] LNet: 325446:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10665.203254] LNetError: 325446:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10665.217241] LNet: Removed LNI 192.168.201.135@tcp [10666.100595] Key type .llcrypt unregistered [10666.102138] Key type ._llcrypt unregistered [10693.115560] Key type ._llcrypt registered [10693.119370] Key type .llcrypt registered [10693.308860] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [10695.058850] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10695.160570] alg: No test for adler32 (adler32-zlib) [10696.320676] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [10696.553363] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [10698.223180] Key type lgssc registered [10699.706898] Lustre: Echo OBD driver; http://www.lustre.org/ [10707.357167] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10707.369976] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10708.756567] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10711.751353] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [10718.194247] LustreError: 326871:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10718.227482] LustreError: 326871:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10718.632310] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10723.297302] LustreError: 326870:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10726.275744] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10726.569491] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10731.994505] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [10742.348172] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10742.769846] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10749.575872] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [10751.986963] LustreError: 328549:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10753.028823] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10758.116960] LustreError: 328551:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10758.145478] LustreError: 328551:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10758.716577] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10763.242155] LustreError: 328551:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10763.270371] LustreError: 328551:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10765.930816] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10766.200532] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10771.459708] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10773.892965] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [10782.419492] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [10788.045688] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10788.232721] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10889.188048] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10889.193794] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10889.196774] LustreError: Skipped 1 previous similar message [10889.205688] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10889.224681] Lustre: Skipped 1 previous similar message [10894.306402] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10894.320047] Lustre: Skipped 2 previous similar messages [10894.815611] Lustre: server umount lustre-OST0001 complete [10899.429761] LustreError: 328552:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10899.455707] LustreError: 328552:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10901.473654] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10901.489898] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10901.511145] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10904.578219] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10904.582643] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10909.670328] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10909.676175] Lustre: Skipped 1 previous similar message [10915.298111] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10915.561398] Lustre: server umount lustre-OST0000 complete [10919.906054] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10919.915580] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10919.924134] Lustre: Skipped 3 previous similar messages [10925.660139] Lustre: server umount lustre-MDT0000 complete [10929.595230] LustreError: 329569:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781283606 with bad export cookie 11543580193774605836 [10929.595863] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10929.608598] LustreError: 329569:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10930.012459] Lustre: server umount lustre-MDT0001 complete [10942.607257] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [10945.709628] Key type lgssc unregistered [10946.030778] LNet: 331814:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10946.035531] LNetError: 331814:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10947.045704] LNet: Removed LNI 192.168.201.135@tcp [10947.865091] Key type .llcrypt unregistered [10947.868909] Key type ._llcrypt unregistered [10994.398958] Key type ._llcrypt registered [10994.403517] Key type .llcrypt registered [10994.518715] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [10995.596923] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10995.635765] alg: No test for adler32 (adler32-zlib) [10996.729373] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [10997.048752] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [10998.799302] Key type lgssc registered [10999.985647] Lustre: Echo OBD driver; http://www.lustre.org/ [11010.220234] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11010.233043] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11011.833525] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11016.759863] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [11021.286032] LustreError: 334119:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11021.296648] LustreError: 334119:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11025.036966] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11025.289382] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11030.340412] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [11034.157620] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11042.221368] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11042.638761] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11050.842757] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [11050.864211] LustreError: 335972:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11055.989511] LustreError: 335972:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11056.002708] LustreError: 335972:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11056.010058] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11060.111949] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11060.400932] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11065.845842] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11067.232555] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [11077.536647] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11086.630949] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11101.674259] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11101.681171] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11101.705461] Lustre: Skipped 3 previous similar messages [11101.725266] Lustre: Skipped 3 previous similar messages [11103.860439] Lustre: server umount lustre-MDT0000 complete [11106.800029] LustreError: 334120:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11106.815816] LustreError: 334120:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11106.914869] LustreError: 334101:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781283783 with bad export cookie 4254856401081034888 [11106.920860] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11106.925269] LustreError: 334101:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11107.303042] Lustre: server umount lustre-MDT0001 complete [11120.250894] Lustre: server umount lustre-OST0000 complete [11134.208090] Lustre: server umount lustre-OST0001 complete [11151.551629] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [11164.487236] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11165.395227] LustreError: 339386:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11165.568589] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11172.955481] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [11175.916485] LustreError: 339387:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11175.930666] LustreError: 339387:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11182.831480] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11183.467553] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11191.143943] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [11196.230896] Lustre: 340526:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11205.693538] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11206.415636] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11208.494844] LustreError: 340880:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11208.528315] LustreError: 340880:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11211.597420] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11213.853570] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [11222.272663] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11228.187786] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11228.646347] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [11235.446806] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11238.992840] Lustre: 342395:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11243.497959] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11243.520407] Lustre: Skipped 2 previous similar messages [11243.528469] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11243.532783] Lustre: Skipped 2 previous similar messages [11248.615653] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11248.618353] Lustre: Skipped 3 previous similar messages [11249.575443] Lustre: server umount lustre-MDT0000 complete [11252.758445] LustreError: 339368:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781283929 with bad export cookie 4254856401081036582 [11252.761380] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11252.766715] LustreError: 339368:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11253.045570] Lustre: server umount lustre-MDT0001 complete [11266.141272] Lustre: server umount lustre-OST0000 complete [11269.087307] Lustre: 333380:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781283930/real 1781283930] req@ffff9bf15da60a80 x1867811309866752/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781283946 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11269.161533] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11269.197662] Lustre: Skipped 1 previous similar message [11270.411509] Lustre: server umount lustre-OST0001 complete [11274.329992] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11276.148703] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11278.051508] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 13:05:53 (1781283953) [11307.214991] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [11318.988176] loop0: detected capacity change from 0 to 1099511627776 [11319.314349] SGI XFS with ACLs, security attributes, quota, no debug enabled [11319.319329] *** VALIDATE xfs *** [11319.324148] XFS (loop0): Mounting V5 Filesystem [11319.337376] XFS (loop0): Ending clean mount [11320.614543] loop1: detected capacity change from 0 to 18691697672192 [11433.279506] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [11443.248403] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [11485.140518] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11485.549211] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11485.578964] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11485.713446] Lustre: lustre-MDT0000: new disk, initializing [11485.862724] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11485.869569] Lustre: Skipped 1 previous similar message [11485.896340] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11490.968618] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [11502.312191] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11502.480541] Lustre: 347533:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11502.540763] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11502.544126] Lustre: Skipped 1 previous similar message [11502.597105] Lustre: lustre-MDT0001: new disk, initializing [11502.694664] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11502.699657] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11507.668682] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [11515.701662] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11521.414809] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11528.741047] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11528.953569] Lustre: lustre-OST0000: new disk, initializing [11528.958517] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11528.965327] Lustre: 349503:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11530.768606] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11530.793222] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11530.958539] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11536.849412] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [11547.665199] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11553.443533] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11553.669587] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11559.165649] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11559.364239] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11561.441437] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11561.452990] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11561.477351] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11561.954387] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11567.077259] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11567.090673] Lustre: Skipped 1 previous similar message [11567.789710] Lustre: server umount lustre-OST0000 complete [11572.193707] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11572.216048] Lustre: Skipped 1 previous similar message [11577.315950] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11577.330213] Lustre: Skipped 4 previous similar messages [11578.020376] Lustre: server umount lustre-MDT0000 complete [11581.701046] LustreError: 347526:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781284258 with bad export cookie 4254856401081038136 [11581.707025] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11581.717767] LustreError: 347526:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11582.046515] Lustre: server umount lustre-MDT0001 complete [11589.269831] XFS (loop0): Unmounting Filesystem [11594.636630] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 13:11:09 (1781284269) [11601.248374] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11601.629264] LustreError: 352062:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11601.641057] LustreError: 352062:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [11601.681277] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11601.686028] Lustre: Skipped 2 previous similar messages [11606.031694] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [11614.711648] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11620.633908] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [11629.177161] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11635.927447] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11642.919670] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11649.119869] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [11657.409721] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11669.985091] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11669.993278] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11670.010268] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11670.037693] Lustre: Skipped 2 previous similar messages [11670.048688] Lustre: Skipped 1 previous similar message [11674.258195] Lustre: server umount lustre-OST0000 complete [11684.137221] Lustre: server umount lustre-MDT0000 complete [11685.350312] LustreError: 352057:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11685.381578] LustreError: 352057:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11687.272569] LustreError: 354650:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781284364 with bad export cookie 4254856401081039025 [11687.283211] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11687.284177] LustreError: 354650:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11687.638415] Lustre: server umount lustre-MDT0001 complete [11700.770824] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [11703.397461] Key type lgssc unregistered [11703.718905] LNet: 355689:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11703.725053] LNetError: 355689:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [11704.745521] LNet: Removed LNI 192.168.201.135@tcp [11705.812504] Key type .llcrypt unregistered [11705.817178] Key type ._llcrypt unregistered [11738.085815] Key type ._llcrypt registered [11738.089311] Key type .llcrypt registered [11738.221307] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [11739.805969] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11739.849205] alg: No test for adler32 (adler32-zlib) [11741.052295] Lustre: Lustre: Build Version: 2.17.54_4_gd2567f2 [11741.364160] LNet: Added LNI 192.168.201.135@tcp [8/256/0/180] [11743.199262] Key type lgssc registered [11744.901863] Lustre: Echo OBD driver; http://www.lustre.org/ [11753.536570] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11753.562657] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11754.976045] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11759.098889] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [11764.195391] LustreError: 357259:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11764.212777] LustreError: 357259:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11767.457602] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11767.809868] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11771.865294] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [11779.522483] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11785.580813] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11791.984724] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11792.260062] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11798.414203] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [11804.804237] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11812.714227] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.35@tcp (stopping) [11813.863967] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11813.870164] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11813.885803] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11814.881642] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11814.897453] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11814.899376] Lustre: Skipped 1 previous similar message [11817.795346] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.35@tcp (stopping) [11818.349889] Lustre: server umount lustre-MDT0001 complete [11820.010524] LustreError: 357259:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11820.034739] LustreError: 357259:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11822.922724] LustreError: 357255:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.35@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11825.128238] LustreError: 357987:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11830.075401] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11830.150857] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11830.157706] Lustre: Found index 1 for lustre-MDT0001, updating log [11830.213255] Lustre: 357245:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11830.247105] LustreError: 357255:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11830.263358] LustreError: 357255:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11830.717494] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11835.393087] LustreError: 360666:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11835.409878] LustreError: 360666:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11835.426269] LustreError: 357251:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11835.560021] LustreError: 360676:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.135@tcp: cfg command failed: rc = -17 [11835.579906] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11835.579906] [11835.889183] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11835.916578] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [12143.074561] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12143.082141] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12143.097894] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12143.112122] Lustre: Skipped 1 previous similar message [12143.125382] Lustre: Skipped 1 previous similar message [12148.159360] Lustre: server umount lustre-MDT0001 complete [12148.209409] LustreError: 357253:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12148.242152] LustreError: 357253:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [12159.689873] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12159.855891] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12159.861192] Lustre: Found index 1 for lustre-MDT0001, updating log [12159.905816] Lustre: 357246:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12160.199201] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12165.277795] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [12165.633234] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12166.646265] LustreError: 369100:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12166.652318] LustreError: 369100:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12166.657763] LustreError: 357251:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12166.660945] LustreError: 357251:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12166.742924] LustreError: 369103:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12166.750609] LustreError: 369103:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12166.758071] LustreError: 369103:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.135@tcp: cfg command failed: rc = -17 [12166.774172] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.135@tcp [12166.774172] [12472.802090] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12472.804845] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12472.820711] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12472.830406] Lustre: Skipped 2 previous similar messages [12477.922893] LustreError: 363845:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12477.938556] LustreError: 363845:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [12478.104941] Lustre: server umount lustre-MDT0001 complete [12491.107953] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12491.172798] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12491.179898] Lustre: Found index 1 for lustre-MDT0001, updating log [12491.205456] Lustre: 357246:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12491.498628] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12496.338290] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [12496.898564] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12499.459739] LustreError: 377458:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12499.464870] LustreError: 377458:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12499.483203] LustreError: 357251:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12499.490287] LustreError: 357251:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12499.625962] LustreError: 377461:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12499.639901] LustreError: 377461:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.135@tcp: cfg command failed: rc = -114 [12499.652387] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12499.652387] [12819.433166] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12819.438880] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12819.455938] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12819.455950] Lustre: Skipped 1 previous similar message [12819.485777] Lustre: Skipped 3 previous similar messages [12825.554776] Lustre: server umount lustre-MDT0000 complete [12829.575960] LustreError: 377044:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781285506 with bad export cookie 17861835815741499802 [12829.584744] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12829.597878] LustreError: 377044:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12829.666356] LustreError: 357254:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12829.677243] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12829.684550] LustreError: 357254:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [12836.093475] Lustre: server umount lustre-MDT0001 complete [12841.013122] Lustre: server umount lustre-OST0000 complete [12853.394313] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [12869.691628] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [12906.755089] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 13:33:02 (1781285582) [12913.799773] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12914.057894] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12914.098844] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12914.212581] Lustre: lustre-MDT0000: new disk, initializing [12914.353935] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12914.390420] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12917.998392] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [12930.515733] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12930.711134] Lustre: 387463:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12930.768586] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12930.774513] Lustre: Skipped 1 previous similar message [12930.868982] Lustre: lustre-MDT0001: new disk, initializing [12931.037055] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12931.123600] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12931.148595] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12938.121275] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [12950.308839] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12958.261611] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12967.142318] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12967.543891] Lustre: lustre-OST0000: new disk, initializing [12967.548132] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12967.558588] Lustre: 389430:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12967.690918] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12974.161545] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12974.172245] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12974.321205] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12975.142629] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_default_debug -1 all [12984.194426] Lustre: DEBUG MARKER: oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12993.060687] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [12993.063334] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12993.070315] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12993.075275] Lustre: Skipped 7 previous similar messages [12999.763269] Lustre: server umount lustre-MDT0000 complete [13000.166369] LustreError: 387469:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13000.242681] LustreError: 387469:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [13003.452225] LustreError: 387453:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781285680 with bad export cookie 17861835815741501755 [13003.452965] LustreError: MGC192.168.201.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13003.455602] LustreError: 388269:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [13003.457897] LustreError: 387453:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [13003.800266] Lustre: server umount lustre-MDT0001 complete [13018.949163] Lustre: server umount lustre-OST0000 complete [13052.754651] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing set_hostid [13063.949340] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing load_modules_local [13097.105508] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12798 sec ========= 13:36:12 (1781285772) [13099.228996] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 13:36:14 (1781285774) === [13103.400404] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 13:36:18 (1781285778) === [13148.373736] Lustre: DEBUG MARKER: oleg135-server.virtnet: executing unload_modules_local [13151.140860] Key type lgssc unregistered [13151.430009] LNet: 395798:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13151.433810] LNetError: 395798:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [13151.453330] LNet: Removed LNI 192.168.201.135@tcp [13152.368819] Key type .llcrypt unregistered [13152.371291] Key type ._llcrypt unregistered