[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 564625558 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.002173] x2apic enabled [ 0.003006] Switched APIC routing to physical x2apic. [ 0.004009] kvm-guest: setup PV IPIs [ 0.005980] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.006000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.006020] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.007009] pid_max: default: 32768 minimum: 301 [ 0.008125] LSM: Security Framework initializing [ 0.009042] Yama: becoming mindful. [ 0.010005] SELinux: Initializing. [ 0.010751] *** VALIDATE selinux *** [ 0.018579] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.022771] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.023129] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.024089] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.025084] *** VALIDATE tmpfs *** [ 0.026403] *** VALIDATE proc *** [ 0.027212] *** VALIDATE cgroup *** [ 0.028005] *** VALIDATE cgroup2 *** [ 0.029244] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.030124] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.031004] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.032049] Spectre V2 : User space: Vulnerable [ 0.033004] Speculative Store Bypass: Vulnerable [ 0.035880] debug: unmapping init [mem 0xffffffff99259000-0xffffffff99260fff] [ 0.037254] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.038614] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.039017] ... version: 2 [ 0.039934] ... bit width: 48 [ 0.040009] ... generic registers: 4 [ 0.040883] ... value mask: 0000ffffffffffff [ 0.041008] ... max period: 00007fffffffffff [ 0.042008] ... fixed-purpose events: 3 [ 0.042850] ... event mask: 000000070000000f [ 0.043306] rcu: Hierarchical SRCU implementation. [ 0.045412] smp: Bringing up secondary CPUs ... [ 0.046522] x86: Booting SMP configuration: [ 0.047014] .... node #0, CPUs: #1 #2 #3 [ 0.050313] smp: Brought up 1 node, 4 CPUs [ 0.052010] smpboot: Max logical packages: 1 [ 0.053014] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.125228] node 0 deferred pages initialised in 69ms [ 0.128096] devtmpfs: initialized [ 0.129246] x86/mm: Memory block size: 128MB [ 0.132512] gcov: version magic: 0x41383552 [ 0.134284] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.137055] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.138234] pinctrl core: initialized pinctrl subsystem [ 0.139141] [ 0.139578] ************************************************************* [ 0.140010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.141009] ** ** [ 0.142010] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.143010] ** ** [ 0.144010] ** This means that this kernel is built to expose internal ** [ 0.145010] ** IOMMU data structures, which may compromise security on ** [ 0.146010] ** your system. ** [ 0.147010] ** ** [ 0.148013] ** If you see this message and you are not debugging the ** [ 0.149011] ** kernel, report this immediately to your vendor! ** [ 0.150014] ** ** [ 0.151014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.152016] ************************************************************* [ 0.153887] NET: Registered protocol family 16 [ 0.154449] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.155058] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.156130] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.158034] cpuidle: using governor menu [ 0.159876] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.162594] PCI: Using configuration type 1 for base access [ 0.164173] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.174064] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.176059] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.181029] cryptd: max_cpu_qlen set to 1000 [ 0.184229] ACPI: Added _OSI(Module Device) [ 0.186010] ACPI: Added _OSI(Processor Device) [ 0.187008] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.188008] ACPI: Added _OSI(Processor Aggregator Device) [ 0.193298] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.199405] ACPI: Interpreter enabled [ 0.201047] ACPI: PM: (supports S0 S3 S4 S5) [ 0.202009] ACPI: Using IOAPIC for interrupt routing [ 0.204118] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.207487] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.217833] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.220036] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.222015] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.225074] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.230374] acpiphp: Slot [2] registered [ 0.232123] acpiphp: Slot [5] registered [ 0.233063] acpiphp: Slot [6] registered [ 0.235128] acpiphp: Slot [7] registered [ 0.236086] acpiphp: Slot [8] registered [ 0.237098] acpiphp: Slot [9] registered [ 0.239117] acpiphp: Slot [10] registered [ 0.240085] acpiphp: Slot [3] registered [ 0.242089] acpiphp: Slot [4] registered [ 0.243113] acpiphp: Slot [11] registered [ 0.245097] acpiphp: Slot [12] registered [ 0.246084] acpiphp: Slot [13] registered [ 0.248077] acpiphp: Slot [14] registered [ 0.249063] acpiphp: Slot [15] registered [ 0.251078] acpiphp: Slot [16] registered [ 0.252100] acpiphp: Slot [17] registered [ 0.254096] acpiphp: Slot [18] registered [ 0.255072] acpiphp: Slot [19] registered [ 0.257069] acpiphp: Slot [20] registered [ 0.258067] acpiphp: Slot [21] registered [ 0.259063] acpiphp: Slot [22] registered [ 0.260091] acpiphp: Slot [23] registered [ 0.262076] acpiphp: Slot [24] registered [ 0.263138] acpiphp: Slot [25] registered [ 0.264085] acpiphp: Slot [26] registered [ 0.266095] acpiphp: Slot [27] registered [ 0.267112] acpiphp: Slot [28] registered [ 0.268065] acpiphp: Slot [29] registered [ 0.270155] acpiphp: Slot [30] registered [ 0.271090] acpiphp: Slot [31] registered [ 0.273056] PCI host bridge to bus 0000:00 [ 0.274016] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.277016] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.278013] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.281021] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.283048] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.285023] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.287223] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.290188] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.293524] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.302761] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.307067] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.310016] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.312012] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.315021] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.319127] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.321835] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.324031] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.326774] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.331011] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.345013] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.349014] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.354067] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.360017] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.366017] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.381021] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.395237] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.416015] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.425014] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.447014] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.459235] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.496043] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.539026] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.622014] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.634697] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.676015] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.712016] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.755015] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.767077] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.810020] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.823016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.860025] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.878508] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.892015] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.908022] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.957016] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.981881] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.984534] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.986263] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.988429] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.991167] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.995239] iommu: Default domain type: Passthrough [ 0.997361] SCSI subsystem initialized [ 0.998103] ACPI: bus type USB registered [ 0.999075] usbcore: registered new interface driver usbfs [ 1.000102] usbcore: registered new interface driver hub [ 1.001064] usbcore: registered new device driver usb [ 1.003166] pps_core: LinuxPPS API ver. 1 registered [ 1.005009] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 1.008059] PTP clock support registered [ 1.010156] EDAC MC: Ver: 3.0.0 [ 1.011504] PCI: Using ACPI for IRQ routing [ 1.013958] NetLabel: Initializing [ 1.016012] NetLabel: domain hash size = 128 [ 1.017009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 1.019085] NetLabel: unlabeled traffic allowed by default [ 1.022098] vgaarb: loaded [ 1.023285] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 1.025012] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 1.032000] clocksource: Switched to clocksource kvm-clock [ 1.129579] VFS: Disk quotas dquot_6.6.0 [ 1.130829] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.132457] *** VALIDATE ramfs *** [ 1.133362] *** VALIDATE hugetlbfs *** [ 1.134507] pnp: PnP ACPI init [ 1.136970] pnp: PnP ACPI: found 6 devices [ 1.181754] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.184422] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.186451] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.188278] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.190597] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.192807] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.194804] NET: Registered protocol family 2 [ 1.196530] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.201234] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.204132] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.208797] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.211667] TCP: Hash tables configured (established 65536 bind 65536) [ 1.214136] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.216471] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.218851] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.220947] NET: Registered protocol family 1 [ 1.223765] RPC: Registered named UNIX socket transport module. [ 1.225955] RPC: Registered udp transport module. [ 1.227821] RPC: Registered tcp transport module. [ 1.229347] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.230970] NET: Registered protocol family 44 [ 1.232267] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.234459] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.236626] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.239127] PCI: CLS 0 bytes, default 64 [ 1.240888] Unpacking initramfs... [ 2.627232] debug: unmapping init [mem 0xffff916dbcc54000-0xffff916dbffbffff] [ 2.630017] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.631295] software IO TLB: mapped [mem 0x00000000a0c00000-0x00000000a4c00000] (64MB) [ 2.633045] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.183273] Initialise system trusted keyrings [ 3.184338] Key type blacklist registered [ 3.186262] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.193720] zbud: loaded [ 3.196416] *** VALIDATE nfs *** [ 3.197336] *** VALIDATE nfs4 *** [ 3.199848] pstore: using deflate compression [ 3.203452] Platform Keyring initialized [ 3.319921] NET: Registered protocol family 38 [ 3.321310] Key type asymmetric registered [ 3.322469] Asymmetric key parser 'x509' registered [ 3.324098] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.327338] io scheduler mq-deadline registered [ 3.328435] io scheduler kyber registered [ 3.329574] io scheduler bfq registered [ 3.331258] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.334057] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.336484] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.338848] ACPI: Power Button [PWRF] [ 3.344308] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.350442] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.361735] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.368197] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.382167] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.408604] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.434855] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.438317] Non-volatile memory driver v1.3 [ 3.439362] Linux agpgart interface v0.103 [ 3.466132] virtio_blk virtio1: [vda] 145904 512-byte logical blocks (74.7 MB/71.2 MiB) [ 3.468646] vda: detected capacity change from 0 to 74702848 [ 3.488178] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.490598] vdb: detected capacity change from 0 to 1073741824 [ 3.503760] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.505836] vdc: detected capacity change from 0 to 2621440000 [ 3.519136] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.521092] vdd: detected capacity change from 0 to 2621440000 [ 3.532906] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.535071] vde: detected capacity change from 0 to 4294967296 [ 3.554709] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.558989] vdf: detected capacity change from 0 to 4294967296 [ 3.598581] libphy: Fixed MDIO Bus: probed [ 3.614383] usbcore: registered new interface driver usbserial_generic [ 3.616298] usbserial: USB Serial support registered for generic [ 3.618150] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.621460] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.622886] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.625568] mousedev: PS/2 mouse device common for all mice [ 3.628156] rtc_cmos 00:05: RTC can wake from S4 [ 3.630580] rtc_cmos 00:05: registered as rtc0 [ 3.631914] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.633995] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.634059] intel_pstate: CPU model not supported [ 3.639884] hid: raw HID events driver (C) Jiri Kosina [ 3.641446] usbcore: registered new interface driver usbhid [ 3.642415] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.643591] usbhid: USB HID core driver [ 3.643747] drop_monitor: Initializing network drop monitor service [ 3.649024] Initializing XFRM netlink socket [ 3.650494] NET: Registered protocol family 10 [ 3.650654] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.652768] Segment Routing with IPv6 [ 3.655132] NET: Registered protocol family 17 [ 3.657022] mpls_gso: MPLS GSO support [ 3.662741] RAS: Correctable Errors collector initialized. [ 3.664102] AVX version of gcm_enc/dec engaged. [ 3.665210] AES CTR mode by8 optimization enabled [ 3.749450] sched_clock: Marking stable (3749424100, 0)->(4575308629, -825884529) [ 3.753217] registered taskstats version 1 [ 3.755506] Loading compiled-in X.509 certificates [ 3.756863] zswap: loaded using pool lzo/zbud [ 3.779167] Key type big_key registered [ 3.791045] Key type encrypted registered [ 3.792740] ima: No TPM chip found, activating TPM-bypass! [ 3.794438] ima: Allocated hash algorithm: sha1 [ 3.795838] ima: No architecture policies found [ 3.797484] evm: Initialising EVM extended attributes: [ 3.798935] evm: security.selinux [ 3.799800] evm: security.ima [ 3.800545] evm: security.capability [ 3.801452] evm: HMAC attrs: 0x1 [ 3.803262] rtc_cmos 00:05: setting system clock to 2026-07-30 06:34:00 UTC (1785393240) [ 3.808108] debug: unmapping init [mem 0xffffffff9a203000-0xffffffff9a3fffff] [ 3.810985] debug: unmapping init [mem 0xffffffff98f82000-0xffffffff99258fff] [ 3.816057] Write protecting the kernel read-only data: 28672k [ 3.818763] debug: unmapping init [mem 0xffffffff97603000-0xffffffff977fffff] [ 3.820904] debug: unmapping init [mem 0xffffffff97f14000-0xffffffff97ffffff] [ 3.849210] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.855193] systemd[1]: Detected virtualization kvm. [ 3.856444] systemd[1]: Detected architecture x86-64. [ 3.857685] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.890394] systemd[1]: No hostname configured. [ 3.891419] systemd[1]: Set hostname to . [ 3.892801] random: systemd: uninitialized urandom read (16 bytes read) [ 3.894484] systemd[1]: Initializing machine ID from random generator. [ 3.931656] random: ln: uninitialized urandom read (6 bytes read) [ 4.005234] random: systemd: uninitialized urandom read (16 bytes read) [ 4.007620] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 4.011497] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 4.014992] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Slices. Starting Setup Virtual Console... [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Starting Create Volatile Files and Directories... Starting Apply Kernel Variables... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Started Memstrack Anylazing Service. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.544094] device-mapper: uevent: version 1.0.3 [ 4.546364] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.222115] virtio_net virtio0 ens2: renamed from eth0 [ 5.223689] random: fast init done [ 5.277713] scsi host0: ata_piix [ 5.284655] scsi host1: ata_piix [ 5.286349] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.289078] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.668848] dracut-initqueue[588]: RTNETLINK answers: File exists [ 10.308277] random: crng init done [ 10.309379] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.617908] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.758963] printk: systemd: 25 output lines suppressed due to ratelimiting [ 12.005345] SELinux: Disabled at runtime. [ 12.065222] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.073668] systemd[1]: Detected virtualization kvm. [ 12.075657] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.553548] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.556532] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.560673] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.564259] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.567451] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.574799] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.581732] systemd[1]: Mounting POSIX Message Queue File System... Mounting POSIX Message Queue File System... [ OK ] Listening on Process Core Dump Socket. [ OK ] Stopped target Switch Root. [ OK ] Reached target rpc_pipefs.target. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Initrd Root File System. Activating swap /dev/disk/by-label/SWAP... Mounting Huge Pages File System... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on udev Kernel Socket. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Created slice system-getty.slice. [ OK [[ 12.667900] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS 0m] Created slice system-sshd\x2dkeygen.slice. Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... [ OK ] Listening on udev Control Socket. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Stopped target Initrd File Systems. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Starting Remount Root and Kernel File Systems... Mounting Kernel Debug File System... Starting udev Coldplug all Devices... [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 13.029366] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Coldplug all Devices. [ OK ] Started udev Kernel Device Manager. [ 13.369473] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.388472] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.544078] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.554937] EDAC sbridge: Ver: 1.1.2 [ 15.155190] Key type dns_resolver registered [ 15.466232] NFS: Registering the id_resolver key type [ 15.468086] Key type id_resolver registered [ 15.469240] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target sshd-keygen.target. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg234-server login: [ 121.001221] hrtimer: interrupt took 3207207 ns [ 145.935794] libcfs: loading out-of-tree module taints kernel. [ 146.102524] Key type ._llcrypt registered [ 146.105556] Key type .llcrypt registered [ 146.309977] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 174.684607] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 176.987487] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 176.996472] alg: No test for adler32 (adler32-zlib) [ 179.315922] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [ 180.824816] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 182.721541] Key type lgssc registered [ 185.435672] Lustre: Echo OBD driver; http://www.lustre.org/ [ 212.856377] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 285.213368] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 308.145453] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 308.207307] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 309.967335] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 310.080212] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 310.321638] Lustre: lustre-MDT0000: new disk, initializing [ 310.540733] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 310.600437] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 317.977409] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 336.628023] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 336.813867] Lustre: 6535:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 336.876416] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 336.880481] Lustre: Skipped 1 previous similar message [ 336.987936] Lustre: lustre-MDT0001: new disk, initializing [ 337.055541] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 337.085519] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 337.094320] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 343.000682] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 348.582771] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 360.328920] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 360.661744] Lustre: lustre-OST0000: new disk, initializing [ 360.669886] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 360.683965] Lustre: 8474:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 360.780461] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 363.439195] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 363.446449] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 363.533909] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 368.999879] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 388.931454] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 389.147270] Lustre: lustre-OST0001: new disk, initializing [ 389.154198] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 389.162580] Lustre: 9551:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 389.257720] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 397.360261] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 397.370182] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 397.500287] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 398.056359] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 413.237093] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 422.016625] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 430.313585] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing check_logdir /tmp/testlogs/ [ 438.745518] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing yml_node [ 446.719296] Lustre: DEBUG MARKER: Client: 2.17.56.2 [ 451.609411] Lustre: DEBUG MARKER: MDS: 2.17.56.2 [ 456.334111] Lustre: DEBUG MARKER: OSS: 2.17.56.2 [ 458.670716] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Jul 30 02:41:33 EDT 2026 [ 482.061436] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 484.162379] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 505.316376] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 505.321800] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 505.339650] Lustre: Skipped 3 previous similar messages [ 505.364253] Lustre: Skipped 3 previous similar messages [ 508.619559] Lustre: server umount lustre-MDT0000 complete [ 514.331574] LustreError: 6526:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785393751 with bad export cookie 12064268884242071403 [ 514.332136] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 514.352077] LustreError: 6526:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 515.554867] LustreError: 6541:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 515.556564] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 515.589429] LustreError: 6541:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 515.600234] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 515.615861] Lustre: Skipped 1 previous similar message [ 515.666778] Lustre: Skipped 1 previous similar message [ 519.585367] LustreError: 6542:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 519.586797] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 519.631616] Lustre: Skipped 1 previous similar message [ 520.838561] Lustre: server umount lustre-MDT0001 complete [ 525.965587] Lustre: server umount lustre-OST0000 complete [ 530.560843] Lustre: server umount lustre-OST0001 complete [ 539.792620] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 552.865783] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 603.523366] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 603.939045] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 603.990319] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 604.129193] Lustre: lustre-MDT0000: new disk, initializing [ 604.338305] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 604.377627] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 610.130606] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 624.642315] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 624.776417] Lustre: 15105:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 624.840961] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 624.848118] Lustre: Skipped 1 previous similar message [ 624.931581] Lustre: lustre-MDT0001: new disk, initializing [ 625.079833] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 625.128624] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 625.146916] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 630.327581] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 640.760352] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 647.511048] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 658.140794] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 658.462851] Lustre: lustre-OST0000: new disk, initializing [ 658.474436] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 658.487593] Lustre: 17078:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 658.719380] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 660.406201] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 660.423674] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 660.707279] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 669.072860] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 682.689793] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 689.540932] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 689.999459] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 697.050441] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 697.512552] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 701.421028] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 701.442252] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 701.445695] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 701.467217] Lustre: Skipped 1 previous similar message [ 706.551534] LustreError: 17063:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 706.569294] LustreError: 17063:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 706.716796] Lustre: server umount lustre-OST0000 complete [ 712.162861] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 712.182446] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 712.198178] Lustre: Skipped 1 previous similar message [ 712.206679] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 716.769259] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 718.641677] Lustre: server umount lustre-MDT0000 complete [ 721.900601] LustreError: 15984:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 721.947259] LustreError: 15984:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 723.678256] LustreError: 15097:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785393960 with bad export cookie 12064268884242072936 [ 723.688451] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 723.701078] LustreError: 15097:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 724.346722] Lustre: server umount lustre-MDT0001 complete [ 730.626562] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 02:46:05 (1785393965) [ 738.877790] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 739.638797] LustreError: 19347:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 739.756697] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 745.724336] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 750.054272] LustreError: 19348:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 750.076477] LustreError: 19348:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 757.112214] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 764.199088] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 776.605082] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 786.041862] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 794.049770] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 803.754877] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 813.702328] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 817.730757] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 824.291640] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 824.300649] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 824.331538] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 824.338870] Lustre: Skipped 2 previous similar messages [ 830.485581] LustreError: 19347:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 830.506370] LustreError: 19347:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 831.075566] Lustre: server umount lustre-MDT0000 complete [ 837.536152] LustreError: 19329:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785394074 with bad export cookie 12064268884242073818 [ 837.550489] LustreError: 19329:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 837.556767] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 839.082556] Lustre: server umount lustre-MDT0001 complete [ 856.199605] Lustre: server umount lustre-OST0000 complete [ 891.788492] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 904.661623] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 905.280705] LustreError: 23756:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 905.295083] LustreError: 23756:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 905.376563] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 905.383616] Lustre: Skipped 2 previous similar messages [ 911.131362] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 922.480867] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 929.504320] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 933.238234] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 942.782937] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 950.901889] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 962.629587] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 963.024238] Lustre: lustre-OST0001: new disk, initializing [ 963.026522] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 963.033240] Lustre: 26074:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 964.218821] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 964.223124] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 964.390837] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 970.835086] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 982.720652] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 987.089284] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 1005.026884] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1005.033732] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1005.046587] Lustre: Skipped 2 previous similar messages [ 1005.055872] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1005.066960] Lustre: Skipped 3 previous similar messages [ 1011.476796] Lustre: server umount lustre-MDT0000 complete [ 1015.788296] LustreError: 26759:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1015.824294] LustreError: 26759:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 1016.943139] LustreError: 25275:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785394253 with bad export cookie 12064268884242074665 [ 1016.947602] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1016.955431] LustreError: 25275:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1017.914649] Lustre: server umount lustre-MDT0001 complete [ 1035.130905] Lustre: server umount lustre-OST0000 complete [ 1036.960587] Lustre: 3660:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785394257/real 1785394257] req@ffff916d02358a80 x1872120687900032/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785394273 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1038.304273] Lustre: 3659:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785394258/real 1785394258] req@ffff916d02359500 x1872120687900288/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785394274 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1042.103873] Lustre: server umount lustre-OST0001 complete [ 1066.369768] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 1083.884296] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1085.347834] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1085.360960] Lustre: Skipped 3 previous similar messages [ 1090.892880] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1100.740269] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1107.122623] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1110.673690] Lustre: 30247:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1120.987533] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1130.185480] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1145.365824] LustreError: 30606:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1145.428310] LustreError: 30606:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 12 previous similar messages [ 1145.859885] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1157.349511] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1167.442130] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1172.348651] Lustre: 32132:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1183.201479] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1183.215458] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1183.241114] Lustre: Skipped 4 previous similar messages [ 1183.253897] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1183.256491] Lustre: Skipped 7 previous similar messages [ 1187.233706] Lustre: server umount lustre-MDT0000 complete [ 1191.774208] LustreError: 30250:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785394428 with bad export cookie 12064268884242075960 [ 1191.776775] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1191.799265] LustreError: 30250:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1192.786015] Lustre: server umount lustre-MDT0001 complete [ 1209.406598] Lustre: server umount lustre-OST0000 complete [ 1211.365695] Lustre: 3658:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785394432/real 1785394432] req@ffff916e3a50df80 x1872120687963136/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785394448 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1214.725939] Lustre: server umount lustre-OST0001 complete [ 1226.934736] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 02:54:21 (1785394461) [ 1265.360818] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 1278.696051] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1284.773921] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1296.147349] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1304.502300] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1308.416386] Lustre: 36108:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1316.377974] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1324.270862] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1334.372745] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1342.268225] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1351.887204] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1356.238209] Lustre: 37987:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1365.484196] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1365.490336] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1365.510038] Lustre: Skipped 6 previous similar messages [ 1365.534179] Lustre: Skipped 3 previous similar messages [ 1370.309774] Lustre: server umount lustre-MDT0000 complete [ 1374.855884] LustreError: 34938:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785394611 with bad export cookie 12064268884242077066 [ 1374.863435] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1374.866154] LustreError: 34938:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1375.385650] Lustre: server umount lustre-MDT0001 complete [ 1390.396592] Lustre: server umount lustre-OST0000 complete [ 1392.096380] Lustre: 3660:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785394612/real 1785394612] req@ffff916d03c20a80 x1872120688018176/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785394628 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1395.387294] Lustre: server umount lustre-OST0001 complete [ 1407.425228] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 02:57:21 (1785394641) [ 1431.226757] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 1443.820607] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1444.124906] LustreError: 40131:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1444.138837] LustreError: 40131:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 23 previous similar messages [ 1444.179498] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1444.183862] Lustre: Skipped 7 previous similar messages [ 1449.068077] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1459.888202] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1465.869662] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1469.688378] Lustre: 41279:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1477.817273] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1487.228345] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1497.452447] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1508.063832] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1517.542155] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1521.627589] Lustre: 43157:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1522.752662] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1522.755564] Lustre: Skipped 1 previous similar message [ 1526.451163] Lustre: lustre-MDT0000: Not available for connect from 192.168.202.34@tcp (stopping) [ 1526.768033] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1526.778753] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1526.800646] Lustre: Skipped 2 previous similar messages [ 1532.046720] Lustre: server umount lustre-MDT0000 complete [ 1537.352952] LustreError: 41281:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785394774 with bad export cookie 12064268884242078172 [ 1537.360576] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1537.375317] LustreError: 41281:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1544.145759] Lustre: server umount lustre-MDT0001 complete [ 1555.100048] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1562.529088] LustreError: 3657:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff916d024f7100 x1872120688075520/t0(0) o250->MGC192.168.202.134@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1570.212767] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1571.472714] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1581.800256] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1587.705289] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1587.720497] Lustre: Skipped 1 previous similar message [ 1588.335764] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1599.801477] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1601.468242] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1610.408810] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1613.070087] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1623.017265] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1624.991987] Lustre: server umount lustre-MDT0000 complete [ 1630.080686] LustreError: 40112:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785394866 with bad export cookie 12064268884242079264 [ 1630.114235] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1630.620647] Lustre: server umount lustre-MDT0001 complete [ 1646.258691] Lustre: server umount lustre-OST0000 complete [ 1650.158405] Lustre: 3658:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785394870/real 1785394870] req@ffff916e3e28b480 x1872120688116480/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785394886 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1652.622979] Lustre: server umount lustre-OST0001 complete [ 1664.331683] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 03:01:39 (1785394899) [ 1692.053537] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 1704.313199] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1710.188963] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1722.234894] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1729.967555] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1735.134812] Lustre: 48960:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1747.725395] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1757.764695] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1769.777048] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1777.621721] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1789.341631] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1796.470583] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1815.011810] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1815.019828] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1815.035033] Lustre: Skipped 10 previous similar messages [ 1815.045716] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1815.063577] Lustre: Skipped 12 previous similar messages [ 1829.344371] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1830.051868] Lustre: server umount lustre-MDT0000 complete [ 1834.660797] LustreError: 50155:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785395071 with bad export cookie 12064268884242080090 [ 1834.675441] LustreError: 50155:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1834.688753] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1835.483744] Lustre: server umount lustre-MDT0001 complete [ 1851.218051] Lustre: server umount lustre-OST0000 complete [ 1853.289703] Lustre: 3659:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785395073/real 1785395073] req@ffff916e3dc39500 x1872120688187392/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785395089 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1856.856894] Lustre: server umount lustre-OST0001 complete [ 1873.903948] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 03:05:07 (1785395107) [ 1877.244619] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1879.390760] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 03:05:14 (1785395114) [ 1892.720190] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 1904.132922] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 1937.688389] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1938.222827] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1938.241622] Lustre: Skipped 1 previous similar message [ 1938.308529] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1938.477285] Lustre: lustre-MDT0000: new disk, initializing [ 1938.632735] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1945.782375] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1957.718931] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1969.052764] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1969.605878] Lustre: lustre-OST0000: new disk, initializing [ 1969.611287] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1969.619165] Lustre: Skipped 1 previous similar message [ 1969.626266] Lustre: 55073:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1969.754808] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1969.757009] Lustre: Skipped 10 previous similar messages [ 1970.972619] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1971.004426] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1971.048416] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1981.245191] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 1995.208342] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1998.300855] Lustre: DEBUG MARKER: create test files [ 2058.229112] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2062.230933] Lustre: server umount lustre-OST0000 complete [ 2067.170230] Lustre: server umount lustre-MDT0000 complete [ 2082.468165] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 2086.598776] Key type lgssc unregistered [ 2087.061982] LNet: 56708:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2087.073673] LNetError: 56708:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2087.098832] LNet: Removed LNI 192.168.202.134@tcp [ 2088.234142] Key type .llcrypt unregistered [ 2088.237336] Key type ._llcrypt unregistered [ 2125.255362] Key type ._llcrypt registered [ 2125.257846] Key type .llcrypt registered [ 2125.382911] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 2127.242029] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2127.286820] alg: No test for adler32 (adler32-zlib) [ 2128.484175] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [ 2128.831943] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 2130.528131] Key type lgssc registered [ 2131.959191] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2141.098511] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2141.149262] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2142.766379] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2149.109533] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2162.023823] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2173.567791] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2174.101930] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2182.363827] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2187.497392] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 2187.669588] Lustre: 59446:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 2187.686829] Lustre: 59446:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 2187.702512] Lustre: 59446:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2187.716574] Lustre: 59446:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 2187.728831] Lustre: 59446:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2187.740671] Lustre: 59446:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2193.577409] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2199.913508] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 2207.391885] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 2221.548424] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2221.778684] Lustre: server umount lustre-OST0000 complete [ 2227.653542] Lustre: server umount lustre-MDT0000 complete [ 2242.605678] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 2246.392099] Key type lgssc unregistered [ 2246.718570] LNet: 61057:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2246.726485] LNetError: 61057:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2246.752647] LNet: Removed LNI 192.168.202.134@tcp [ 2247.945545] Key type .llcrypt unregistered [ 2247.952839] Key type ._llcrypt unregistered [ 2290.073465] Key type ._llcrypt registered [ 2290.075899] Key type .llcrypt registered [ 2290.170736] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 2291.393555] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2291.471835] alg: No test for adler32 (adler32-zlib) [ 2292.723102] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [ 2293.075354] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 2294.792164] Key type lgssc registered [ 2296.505103] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2305.760247] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2305.788509] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2307.183243] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2312.159742] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2321.590168] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2329.626380] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2330.061226] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2332.160359] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 2332.305364] Lustre: 63900:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 2332.318244] Lustre: 63900:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 2332.329381] Lustre: 63900:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2332.335412] Lustre: 63900:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 2332.351077] Lustre: 63900:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2332.369335] Lustre: 63900:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2337.005216] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2345.825559] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2351.689642] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 2367.970504] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2367.992065] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2368.035678] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2369.314689] Lustre: server umount lustre-OST0000 complete [ 2375.639365] Lustre: server umount lustre-MDT0000 complete [ 2395.545439] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 2400.063095] Key type lgssc unregistered [ 2400.629920] LNet: 65479:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2400.644680] LNetError: 65479:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2401.706974] LNet: Removed LNI 192.168.202.134@tcp [ 2403.826532] Key type .llcrypt unregistered [ 2403.831603] Key type ._llcrypt unregistered [ 2433.857492] Key type ._llcrypt registered [ 2433.860343] Key type .llcrypt registered [ 2434.100822] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 2459.026679] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 2460.824974] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2460.867321] alg: No test for adler32 (adler32-zlib) [ 2462.069427] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [ 2462.570634] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 2464.340067] Key type lgssc registered [ 2466.426816] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2517.623909] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2517.687513] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2519.124250] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2519.182834] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2519.297605] Lustre: lustre-MDT0000: new disk, initializing [ 2519.453451] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2519.482034] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2526.977618] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2542.314729] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2542.515567] Lustre: 68636:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2542.635470] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2542.640586] Lustre: Skipped 1 previous similar message [ 2542.758066] Lustre: lustre-MDT0001: new disk, initializing [ 2542.923632] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2542.978196] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2543.013058] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2548.729362] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2560.759661] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2566.467765] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2577.208886] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2577.636216] Lustre: lustre-OST0000: new disk, initializing [ 2577.639341] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2577.644278] Lustre: 70606:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2577.701797] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2587.833125] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2587.862328] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2588.138189] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2589.226582] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2602.809827] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2608.955503] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2609.247309] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2614.379670] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2614.587249] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2618.851039] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2618.862175] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2618.868513] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2618.872816] Lustre: Skipped 1 previous similar message [ 2623.742730] Lustre: server umount lustre-OST0000 complete [ 2629.094102] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2629.118156] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2629.126489] Lustre: Skipped 2 previous similar messages [ 2634.210909] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2634.233178] Lustre: Skipped 1 previous similar message [ 2636.134129] Lustre: server umount lustre-MDT0000 complete [ 2642.653782] LustreError: 68626:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785395879 with bad export cookie 17806018227168508937 [ 2642.669511] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2642.673958] LustreError: 68626:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2644.217282] Lustre: server umount lustre-MDT0001 complete [ 2659.447660] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 03:18:13 (1785395893) [ 2676.940905] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 2734.214250] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 2757.426155] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 2810.140206] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2810.433213] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2810.492749] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2810.692841] Lustre: lustre-MDT0000: new disk, initializing [ 2810.856078] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2810.890521] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2817.560647] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2834.759384] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2834.945922] Lustre: 76680:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2835.022731] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2835.026156] Lustre: Skipped 1 previous similar message [ 2835.083333] Lustre: lustre-MDT0001: new disk, initializing [ 2835.202359] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2835.257528] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2835.281149] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2843.130912] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2856.278270] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2864.055791] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2876.517236] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2876.897985] Lustre: lustre-OST0000: new disk, initializing [ 2876.900650] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2876.906587] Lustre: 78651:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2876.974636] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2885.293269] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2885.323586] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2885.648870] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2888.072867] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2901.439018] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2907.677735] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2907.945234] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2912.551917] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2912.813976] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2916.321964] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2916.325323] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2916.356876] LustreError: Skipped 1 previous similar message [ 2916.361026] Lustre: Skipped 1 previous similar message [ 2916.378501] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2921.459104] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2921.471388] Lustre: Skipped 2 previous similar messages [ 2930.144426] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2930.547734] Lustre: server umount lustre-OST0000 complete [ 2936.802327] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2936.838906] Lustre: Skipped 1 previous similar message [ 2936.859390] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2936.873403] Lustre: Skipped 3 previous similar messages [ 2942.471647] Lustre: server umount lustre-MDT0000 complete [ 2947.060035] LustreError: 76692:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2947.089896] LustreError: 76692:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2948.450732] LustreError: 77476:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785396185 with bad export cookie 17806018227168509819 [ 2948.462114] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2948.477440] LustreError: 77476:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2949.285940] Lustre: server umount lustre-MDT0001 complete [ 2965.531989] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 03:23:19 (1785396199) [ 2973.336154] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2973.795919] LustreError: 81078:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2973.855217] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2979.299851] LustreError: 81079:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2979.609729] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 2983.403945] LustreError: 81078:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2988.516433] LustreError: 81079:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2991.998671] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3000.205121] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3014.664589] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3024.297934] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3033.562141] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3033.857015] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3033.869426] Lustre: Skipped 1 previous similar message [ 3042.159087] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3050.404995] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3056.335798] Lustre: 83752:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 169ce565-282a-4f25-bf2d-58f1fb6ea77c at adminstrative request [ 3059.709226] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3059.718332] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 3069.940798] Lustre: MGS: Client 169ce565-282a-4f25-bf2d-58f1fb6ea77c (at 0@lo) reconnecting [ 3069.965228] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xf71baf3cbe8282ed to 0xf71baf3cbe82858d [ 3069.991143] Lustre: MGC192.168.202.134@tcp: Connection restored to 0@lo (at 0@lo) [ 3097.093853] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3097.409540] Lustre: lustre-OST0001: new disk, initializing [ 3097.416854] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3097.435791] Lustre: 84150:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3106.536394] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3107.523239] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3107.550765] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3107.731921] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3120.630732] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3128.293475] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3128.298041] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3128.326687] Lustre: Skipped 1 previous similar message [ 3128.339500] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 3128.366241] Lustre: Skipped 3 previous similar messages [ 3139.040136] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3139.685718] Lustre: server umount lustre-OST0001 complete [ 3143.651468] LustreError: 82904:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3143.674241] LustreError: 82904:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3146.721520] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3146.736896] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3146.757849] Lustre: Skipped 1 previous similar message [ 3151.708593] Lustre: server umount lustre-OST0000 complete [ 3163.147237] Lustre: server umount lustre-MDT0000 complete [ 3164.130808] LustreError: 81807:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3164.168379] LustreError: 81807:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3167.590300] LustreError: 81060:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785396404 with bad export cookie 17806018227168511373 [ 3167.594059] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3167.611860] LustreError: 81060:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3168.071400] Lustre: server umount lustre-MDT0001 complete [ 3182.383720] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 03:26:55 (1785396415) [ 3183.846170] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 3186.045482] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 03:27:00 (1785396420) [ 3189.392186] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 3192.898516] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 03:27:06 (1785396426) [ 3196.113963] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 3200.842756] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 03:27:13 (1785396433) [ 3202.698319] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 3205.369554] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 03:27:19 (1785396439) [ 3207.257560] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 3214.882379] loop: module loaded [ 3214.903799] loop0: detected capacity change from 0 to 4294967296 [ 3214.958161] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3216.890144] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 3221.684223] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 3226.602491] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 3262.559708] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3262.643078] LustreError: 88283:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 3262.655919] LustreError: 88283:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 3262.668526] LustreError: 88283:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 3262.690207] LustreError: 88283:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 3289.809592] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 3310.792209] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 3369.071597] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3369.592126] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3369.651892] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3369.860351] Lustre: lustre-MDT0000: new disk, initializing [ 3369.995314] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3370.001671] Lustre: Skipped 1 previous similar message [ 3370.036799] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3377.104554] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3391.173184] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3391.379617] Lustre: 91175:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3391.784287] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3397.652374] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3411.097178] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3420.676710] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3430.705157] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3431.146588] Lustre: 93146:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3437.160988] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3437.175458] Lustre: Skipped 1 previous similar message [ 3437.182832] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3437.250532] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3439.944445] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3455.123711] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3462.558125] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3462.929724] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3468.030901] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3468.309251] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3473.377585] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3473.380253] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3473.403030] LustreError: Skipped 1 previous similar message [ 3473.403672] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3473.403676] Lustre: Skipped 9 previous similar messages [ 3473.467486] Lustre: Skipped 4 previous similar messages [ 3485.152182] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3485.543934] Lustre: server umount lustre-OST0000 complete [ 3496.849671] Lustre: server umount lustre-MDT0000 complete [ 3498.980567] LustreError: 91184:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3498.998811] LustreError: 91184:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3501.838682] LustreError: 91168:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785396738 with bad export cookie 17806018227168511989 [ 3501.845098] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3501.847521] LustreError: 91168:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3502.161785] Lustre: server umount lustre-MDT0001 complete [ 3511.210398] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 03:32:25 (1785396745) [ 3520.652601] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3521.548477] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 3521.562196] Lustre: Skipped 2 previous similar messages [ 3527.042190] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3538.240501] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3546.588591] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3557.491138] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3566.330266] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3580.094611] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3592.145476] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3604.733932] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3613.547607] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3614.153595] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3621.747965] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3622.095225] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3631.716241] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3632.204517] Lustre: lustre-OST0001: new disk, initializing [ 3632.208063] Lustre: Skipped 2 previous similar messages [ 3632.243206] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3632.260633] Lustre: Skipped 3 previous similar messages [ 3632.295272] Lustre: 98939:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3641.095983] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3641.115477] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3641.466783] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3644.683447] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3663.393899] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3674.280414] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 3674.588713] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3681.021643] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 3681.395377] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3705.535285] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3734.588908] Lustre: Failing over lustre-MDT0000 [ 3735.554347] Lustre: server umount lustre-MDT0000 complete [ 3735.716252] LustreError: 95557:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.34@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3735.751772] LustreError: 95557:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 3738.598114] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3738.614973] Lustre: Skipped 4 previous similar messages [ 3754.977571] Lustre: 66860:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785396975/real 1785396975] req@00000000c120dd60 x1872123080803840/t0(0) o400->MGC192.168.202.134@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1785396991 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3755.016574] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3766.622851] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3766.625818] LDISKFS-fs (dm-0): recovery complete [ 3766.659901] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3779.619606] LustreError: 66857:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000d3399add x1872123080815488/t0(0) o250->MGC192.168.202.134@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3780.647982] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 3780.674446] Lustre: Skipped 3 previous similar messages [ 3780.859416] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3781.750404] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3781.755045] Lustre: 95558:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3786.474855] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3786.476525] Lustre: 95562:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3786.529743] Lustre: 95562:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 1 previous similar message [ 3786.772373] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3787.130216] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3787.468381] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3787.497510] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 3787.841578] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3787.862426] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 3787.902739] Lustre: 101791:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3787.947499] Lustre: 101791:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 7 previous similar messages [ 3788.637287] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3788.660660] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 3788.993157] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3789.004121] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 3790.047217] Lustre: 101791:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3790.078924] Lustre: 101791:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 5 previous similar messages [ 3790.770420] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3790.801015] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 5 previous similar messages [ 3791.073466] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3791.098768] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 5 previous similar messages [ 3793.010235] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3794.064864] Lustre: 101791:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3794.081881] Lustre: 101791:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 11 previous similar messages [ 3795.081702] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3795.102880] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 3795.434207] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3795.455240] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 3802.348776] Lustre: 101791:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3802.383620] Lustre: 101791:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 24 previous similar messages [ 3803.385271] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3803.443363] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 24 previous similar messages [ 3803.776234] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3803.792277] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 24 previous similar messages [ 3818.570448] Lustre: 101791:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3818.603330] Lustre: 101791:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 49 previous similar messages [ 3819.588726] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3819.612801] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 49 previous similar messages [ 3819.952103] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3819.975103] LustreError: 101791:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 49 previous similar messages [ 3841.312215] Lustre: 101791:0:(ldlm_lib.c:2393:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3841.329273] Lustre: 101791:0:(ldlm_lib.c:2403:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3841.342253] Lustre: 101791:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client b26beee4-5c97-4b6c-94f0-f27777faf1c8@192.168.202.34@tcp [ 3841.371195] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3841.386875] Lustre: 101791:0:(ldlm_lib.c:1913:abort_req_replay_queue()) @@@ aborted: req@00000000cdeb1ab1 x1872123054328064/t0(8589934763) o101->b26beee4-5c97-4b6c-94f0-f27777faf1c8@192.168.202.34@tcp:268/0 lens 592/0 e 0 to 0 dl 1785397088 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3841.469197] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3841.485622] Lustre: lustre-MDT0000: Denying connection for new client b26beee4-5c97-4b6c-94f0-f27777faf1c8 (at 192.168.202.34@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3841.526114] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3841.637639] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3841.652062] Lustre: Skipped 2 previous similar messages [ 3841.660351] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3841.796475] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:76 to 0x280000401:577) [ 3841.802570] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:43 to 0x2c0000401:545) [ 3866.081373] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3866.101335] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3866.110853] Lustre: Skipped 1 previous similar message [ 3866.121989] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3866.128741] Lustre: Skipped 7 previous similar messages [ 3871.165452] Lustre: server umount lustre-OST0000 complete [ 3873.765143] LustreError: 102345:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3873.787511] LustreError: 102345:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 59 previous similar messages [ 3882.964221] Lustre: server umount lustre-OST0001 complete [ 3895.248671] Lustre: server umount lustre-MDT0000 complete [ 3900.499165] LustreError: 98938:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785397137 with bad export cookie 17806018227168563208 [ 3900.500115] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3900.516124] LustreError: 98938:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3901.178870] Lustre: server umount lustre-MDT0001 complete [ 3911.245367] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 03:39:06 (1785397146) [ 3921.331771] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3921.342508] Lustre: *** cfs_fail_loc=197, val=0*** [ 3937.121383] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3942.240476] LustreError: 103797:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.134@tcp: failed processing log, type 4: rc = -110 [ 3947.394510] Lustre: *** cfs_fail_loc=197, val=0*** [ 3947.396048] Lustre: Skipped 65 previous similar messages [ 3975.182528] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 3986.922844] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3991.161330] Lustre: server umount lustre-OST0000 complete [ 4003.197479] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 03:40:37 (1785397237) [ 4027.711544] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4034.450861] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4046.924326] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4052.121852] LustreError: 105453:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 4052.148359] LustreError: 105453:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 4052.163674] LustreError: 105453:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 23, retries 0, failed: rc = -5 [ 4053.094668] Lustre: server umount lustre-MDT0000 complete [ 4097.501622] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 4111.215982] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 4164.386550] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4164.942262] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4165.175072] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4165.403704] Lustre: lustre-MDT0000: new disk, initializing [ 4165.693628] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4172.640212] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4186.707965] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4186.942258] Lustre: 109634:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4187.600073] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4192.447902] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4204.532621] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4212.422286] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4223.296508] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4223.854354] Lustre: 111607:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4232.038361] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4235.472580] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4248.271473] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4255.119612] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4255.431836] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4262.168691] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4262.498877] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4268.002151] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4268.003509] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4268.026286] LustreError: Skipped 3 previous similar messages [ 4268.047878] Lustre: Skipped 5 previous similar messages [ 4268.058544] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4268.058557] Lustre: Skipped 9 previous similar messages [ 4271.922826] Lustre: server umount lustre-OST0000 complete [ 4285.096149] Lustre: server umount lustre-MDT0000 complete [ 4287.458335] LustreError: 109647:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4287.489074] LustreError: 109647:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 10 previous similar messages [ 4290.438387] LustreError: 109628:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785397527 with bad export cookie 17806018227168573715 [ 4290.445458] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4290.453715] LustreError: 109628:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4291.176959] Lustre: server umount lustre-MDT0001 complete [ 4303.309213] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 03:45:37 (1785397537) [ 4314.309253] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 4319.137944] Key type lgssc unregistered [ 4319.740495] LNet: 114099:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4319.783929] LNetError: 114099:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4320.878875] LNet: Removed LNI 192.168.202.134@tcp [ 4322.931289] Key type .llcrypt unregistered [ 4322.935400] Key type ._llcrypt unregistered [ 4356.108595] Key type ._llcrypt registered [ 4356.117951] Key type .llcrypt registered [ 4356.347400] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 4382.257074] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 4384.347881] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4384.589736] alg: No test for adler32 (adler32-zlib) [ 4385.876810] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [ 4386.128290] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 4387.800227] Key type lgssc registered [ 4389.899671] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4450.412382] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4450.486463] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4452.248651] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4452.338402] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4452.588849] Lustre: lustre-MDT0000: new disk, initializing [ 4452.834456] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4452.881693] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4463.459939] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4482.469944] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4482.952172] Lustre: lustre-OST0000: new disk, initializing [ 4482.960644] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4482.968397] Lustre: Skipped 1 previous similar message [ 4482.973740] Lustre: 118299:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4483.115087] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4488.426916] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4488.434884] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4488.504287] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4491.265664] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4504.738075] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4504.944674] Lustre: lustre-OST0001: new disk, initializing [ 4504.957774] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4504.969654] Lustre: 119165:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4505.099915] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4510.370705] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 4510.386630] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 4510.431076] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 4513.036307] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4526.050449] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4526.053571] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4526.082865] Lustre: Skipped 1 previous similar message [ 4529.779186] Lustre: server umount lustre-MDT0000 complete [ 4537.846679] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 4544.056698] LustreError: 118298:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785397780 with bad export cookie 5479007314710878348 [ 4544.058349] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4544.065949] LustreError: 118298:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4544.489706] Lustre: server umount lustre-OST0000 complete [ 4549.432397] Lustre: server umount lustre-OST0001 complete [ 4561.695526] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 03:49:55 (1785397795) [ 4594.686269] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 03:50:28 (1785397828) [ 4607.123636] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 4639.041894] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 03:51:13 (1785397873) [ 4696.987028] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 4709.376945] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 4769.049828] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4769.524039] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4769.561804] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4769.692483] Lustre: lustre-MDT0000: new disk, initializing [ 4769.873106] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4769.923542] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4776.550048] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4790.793039] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4791.019812] Lustre: 126927:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4791.110988] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4791.118790] Lustre: Skipped 1 previous similar message [ 4791.224972] Lustre: lustre-MDT0001: new disk, initializing [ 4791.400684] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4791.481576] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4791.502340] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4798.170239] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4810.169848] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4818.317997] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4826.983642] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4827.257152] Lustre: lustre-OST0000: new disk, initializing [ 4827.264786] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4827.277743] Lustre: 128899:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4827.419958] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4836.478387] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4836.946535] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4836.964119] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4837.092539] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4848.645486] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4855.926403] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4856.280642] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4862.489992] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4862.831649] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4868.069094] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4868.072550] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4868.076693] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4868.076701] Lustre: Skipped 1 previous similar message [ 4868.081929] LustreError: Skipped 1 previous similar message [ 4868.117791] Lustre: Skipped 1 previous similar message [ 4871.778631] Lustre: server umount lustre-OST0000 complete [ 4878.305327] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4878.308604] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4878.313237] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4878.343388] Lustre: Skipped 1 previous similar message [ 4878.367926] Lustre: Skipped 2 previous similar messages [ 4883.258634] Lustre: server umount lustre-MDT0000 complete [ 4890.862313] LustreError: 126918:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785398127 with bad export cookie 5479007314710879573 [ 4890.883364] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4890.899712] LustreError: 126918:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4891.808321] Lustre: server umount lustre-MDT0001 complete [ 4903.937850] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 03:55:38 (1785398138) [ 4913.151947] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4913.797661] LustreError: 131315:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4913.841478] LustreError: 131315:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 4913.934131] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4919.268871] LustreError: 131316:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4919.898603] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4923.368557] LustreError: 131315:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4928.484784] LustreError: 131316:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4930.759142] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4937.079831] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4947.873884] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4956.376301] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4967.292909] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4967.834878] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4967.846717] Lustre: Skipped 1 previous similar message [ 4975.305941] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 4985.651489] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4992.488369] Lustre: *** cfs_fail_loc=159, val=0*** [ 5008.122587] Lustre: lustre-MDT0000: Client f2db93ec-9b1c-435e-976f-38fa34c8f36e (at 192.168.202.34@tcp) reconnecting [ 5011.001875] Lustre: *** cfs_fail_loc=159, val=0*** [ 5011.007933] Lustre: Skipped 5 previous similar messages [ 5027.578429] Lustre: lustre-MDT0000: Client f2db93ec-9b1c-435e-976f-38fa34c8f36e (at 192.168.202.34@tcp) reconnecting [ 5047.788519] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5047.802943] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5047.839018] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5048.733864] Lustre: server umount lustre-OST0000 complete [ 5055.972282] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5055.986526] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5055.989455] Lustre: Skipped 1 previous similar message [ 5056.007384] Lustre: Skipped 1 previous similar message [ 5066.219182] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5066.229913] Lustre: Skipped 3 previous similar messages [ 5066.600707] Lustre: server umount lustre-MDT0000 complete [ 5071.329679] LustreError: 132411:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5071.337749] LustreError: 132411:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5073.988175] LustreError: 133919:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785398310 with bad export cookie 5479007314710880448 [ 5074.022036] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5074.877937] Lustre: server umount lustre-MDT0001 complete [ 5090.180735] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 5094.269210] Key type lgssc unregistered [ 5094.720283] LNet: 135096:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5094.733566] LNetError: 135096:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5094.762069] LNet: Removed LNI 192.168.202.134@tcp [ 5096.304482] Key type .llcrypt unregistered [ 5096.310550] Key type ._llcrypt unregistered [ 5131.409118] Key type ._llcrypt registered [ 5131.415680] Key type .llcrypt registered [ 5131.640818] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 5134.042915] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5134.165944] alg: No test for adler32 (adler32-zlib) [ 5135.431865] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [ 5135.864525] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 5137.712099] Key type lgssc registered [ 5139.529325] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5150.456710] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5150.490485] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5151.975025] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5157.321102] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5161.447282] LustreError: 136513:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5161.463269] LustreError: 136513:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5166.564645] LustreError: 136514:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5168.840538] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5169.455615] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5176.165524] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5184.999714] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5192.679954] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5202.620118] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5203.136895] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5210.777617] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5220.591048] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5221.580135] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 5237.731941] Lustre: *** cfs_fail_loc=159, val=0*** [ 5254.307057] Lustre: lustre-MDT0000: Client c200822e-46ce-404b-b667-2cc75c027315 (at 192.168.202.34@tcp) reconnecting [ 5258.354721] Lustre: *** cfs_fail_loc=159, val=0*** [ 5258.363565] Lustre: Skipped 1 previous similar message [ 5273.777564] Lustre: lustre-MDT0001: Client c200822e-46ce-404b-b667-2cc75c027315 (at 192.168.202.34@tcp) reconnecting [ 5276.945328] Lustre: *** cfs_fail_loc=159, val=0*** [ 5276.957140] Lustre: Skipped 2 previous similar messages [ 5293.263620] Lustre: lustre-MDT0001: Client c200822e-46ce-404b-b667-2cc75c027315 (at 192.168.202.34@tcp) reconnecting [ 5300.152322] Lustre: 139585:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5302.897168] Lustre: *** cfs_fail_loc=159, val=0*** [ 5302.899805] Lustre: Skipped 2 previous similar messages [ 5318.826204] Lustre: lustre-MDT0000: Client f369338c-084f-466e-bc31-de178ea0c35a (at 192.168.202.34@tcp) reconnecting [ 5321.317242] Lustre: *** cfs_fail_loc=159, val=0*** [ 5321.318873] Lustre: Skipped 14 previous similar messages [ 5336.787272] Lustre: lustre-MDT0000: Client f369338c-084f-466e-bc31-de178ea0c35a (at 192.168.202.34@tcp) reconnecting [ 5339.444716] Lustre: 139878:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5349.863579] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5349.865794] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5349.874569] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5349.896204] Lustre: Skipped 1 previous similar message [ 5349.920974] Lustre: Skipped 1 previous similar message [ 5354.767777] Lustre: server umount lustre-OST0000 complete [ 5364.193406] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5364.217218] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5364.257291] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5365.219262] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5367.588546] Lustre: server umount lustre-MDT0000 complete [ 5369.381671] LustreError: 136513:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5369.398851] LustreError: 136513:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5373.141900] LustreError: 136495:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785398609 with bad export cookie 684300209876570174 [ 5373.146515] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5373.187792] LustreError: 136495:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5374.360194] Lustre: server umount lustre-MDT0001 complete [ 5391.192823] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 5395.815921] Key type lgssc unregistered [ 5396.300525] LNet: 140940:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5396.313049] LNetError: 140940:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5396.354749] LNet: Removed LNI 192.168.202.134@tcp [ 5397.947775] Key type .llcrypt unregistered [ 5397.949481] Key type ._llcrypt unregistered [ 5432.997429] Key type ._llcrypt registered [ 5433.006197] Key type .llcrypt registered [ 5433.157684] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 5434.975120] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5435.149668] alg: No test for adler32 (adler32-zlib) [ 5436.935889] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [ 5437.307108] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 5439.112194] Key type lgssc registered [ 5440.517625] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5451.970344] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5451.999771] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5453.789965] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5459.316754] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5464.037921] LustreError: 142356:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5464.048471] LustreError: 142356:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5469.190322] LustreError: 142355:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5471.852796] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5472.505622] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5478.846373] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5488.720979] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5497.351772] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5507.462105] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5508.064982] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5517.980450] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5518.343859] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 5523.474072] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 5529.277882] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5538.750161] Lustre: 144941:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5543.812396] Lustre: 144989:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5545.906350] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 5547.639815] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 5556.797365] Lustre: 145148:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 5558.796682] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 5560.277891] Lustre: 145249:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5569.518408] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5569.528754] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5569.529601] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5569.549066] Lustre: Skipped 1 previous similar message [ 5569.591648] LustreError: Skipped 1 previous similar message [ 5571.883155] Lustre: server umount lustre-OST0000 complete [ 5579.745863] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5579.748166] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5579.756639] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5579.773636] Lustre: Skipped 1 previous similar message [ 5579.782600] Lustre: Skipped 1 previous similar message [ 5583.666713] Lustre: server umount lustre-MDT0000 complete [ 5584.869831] LustreError: 142356:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5584.902354] LustreError: 142356:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5587.940573] LustreError: 145040:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785398824 with bad export cookie 11903787579362299852 [ 5587.942374] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5587.949362] LustreError: 145040:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5588.488080] Lustre: server umount lustre-MDT0001 complete [ 5603.724133] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 5608.086419] Key type lgssc unregistered [ 5608.466946] LNet: 146309:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5608.484282] LNetError: 146309:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5608.500869] LNet: Removed LNI 192.168.202.134@tcp [ 5610.011901] Key type .llcrypt unregistered [ 5610.015875] Key type ._llcrypt unregistered [ 5642.247496] Key type ._llcrypt registered [ 5642.251373] Key type .llcrypt registered [ 5642.382858] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 5643.776190] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5643.921066] alg: No test for adler32 (adler32-zlib) [ 5645.419240] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [ 5645.903398] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 5647.824167] Key type lgssc registered [ 5649.952516] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5661.979567] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5662.006774] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5663.881741] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5669.952929] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5673.442659] LustreError: 147727:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5673.468425] LustreError: 147727:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5678.570622] LustreError: 147728:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5682.899444] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5683.574804] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5688.475427] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5697.894177] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5706.146279] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5715.255742] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5715.855633] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5723.793531] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5729.214480] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 5732.524324] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5734.340644] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 5744.633826] Lustre: *** cfs_fail_loc=159, val=0*** [ 5761.216463] Lustre: lustre-MDT0000: Client 153d89e9-d211-4f4f-87eb-ac1731801c03 (at 192.168.202.34@tcp) reconnecting [ 5770.209588] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5770.210312] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5770.218925] LustreError: Skipped 1 previous similar message [ 5770.221047] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5770.247153] Lustre: Skipped 1 previous similar message [ 5774.684435] Lustre: server umount lustre-OST0000 complete [ 5780.963834] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5780.974577] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5780.999847] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5781.005079] Lustre: Skipped 1 previous similar message [ 5785.575768] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5785.580129] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5790.690569] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5790.708572] Lustre: Skipped 2 previous similar messages [ 5795.296136] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5795.994018] Lustre: server umount lustre-MDT0000 complete [ 5796.833171] LustreError: 147727:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5796.873239] LustreError: 147727:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5801.953361] LustreError: 150144:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5801.988313] LustreError: 150144:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5803.465377] LustreError: 148469:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785399040 with bad export cookie 12667095446499531667 [ 5803.479165] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5803.487824] LustreError: 148469:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5804.788536] Lustre: server umount lustre-MDT0001 complete [ 5825.609209] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 5831.008171] Key type lgssc unregistered [ 5831.700867] LNet: 151440:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5831.724186] LNetError: 151440:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5831.742382] LNet: Removed LNI 192.168.202.134@tcp [ 5833.134187] Key type .llcrypt unregistered [ 5833.136810] Key type ._llcrypt unregistered [ 5872.270955] Key type ._llcrypt registered [ 5872.272735] Key type .llcrypt registered [ 5872.521283] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 5873.772074] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5873.837538] alg: No test for adler32 (adler32-zlib) [ 5875.558959] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [ 5875.988864] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 5877.785701] Key type lgssc registered [ 5879.582685] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5892.063050] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5892.101837] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5893.898634] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5899.127015] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5903.337537] LustreError: 152855:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5903.368534] LustreError: 152855:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5908.453634] LustreError: 152856:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5912.334615] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5913.056716] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5922.098925] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5932.458441] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5944.362522] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5957.734883] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5958.130565] LustreError: 154683:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5958.581618] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5963.720500] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 5963.732121] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 5967.894945] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 5977.055618] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6000.667657] Lustre: lustre-MDT0001: evicting 3fca0060-f6de-45b3-830a-da418da97d7e (at 192.168.202.34@tcp) by administrative request [ 6000.679176] Lustre: Skipped 1 previous similar message [ 6035.431897] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6035.441384] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6035.455671] Lustre: Skipped 1 previous similar message [ 6035.457248] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6036.686299] Lustre: server umount lustre-OST0000 complete [ 6045.665268] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6045.667966] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6045.686098] Lustre: Skipped 1 previous similar message [ 6045.703397] Lustre: Skipped 2 previous similar messages [ 6048.732912] Lustre: server umount lustre-MDT0000 complete [ 6050.784867] LustreError: 152856:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6050.808091] LustreError: 152856:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6054.744623] LustreError: 152836:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785399291 with bad export cookie 8393919524846230174 [ 6054.767723] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6054.771163] LustreError: 152836:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 6055.398309] Lustre: server umount lustre-MDT0001 complete [ 6070.165281] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 6073.992694] Key type lgssc unregistered [ 6074.359627] LNet: 156934:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6074.367517] LNetError: 156934:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6074.397773] LNet: Removed LNI 192.168.202.134@tcp [ 6075.950062] Key type .llcrypt unregistered [ 6075.956478] Key type ._llcrypt unregistered [ 6194.931131] Key type ._llcrypt registered [ 6194.932392] Key type .llcrypt registered [ 6195.087715] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 6197.640200] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6197.664289] alg: No test for adler32 (adler32-zlib) [ 6199.063373] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [ 6199.466504] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 6201.102861] Key type lgssc registered [ 6202.818598] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6215.073971] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6215.116323] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6216.773297] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6224.332790] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6226.915920] LustreError: 159431:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6226.949214] LustreError: 159431:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6232.033895] LustreError: 159432:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6235.907545] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6236.501301] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6242.433120] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6255.562286] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6267.139968] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6281.875691] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6283.056358] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6287.172943] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 6291.086463] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6301.749677] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6306.622513] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 6322.145589] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6322.150100] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6322.173888] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6322.200296] Lustre: Skipped 1 previous similar message [ 6322.216371] Lustre: Skipped 1 previous similar message [ 6325.428746] Lustre: server umount lustre-OST0000 complete [ 6332.385633] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6332.421012] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6337.515294] LustreError: 161440:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6337.545170] LustreError: 161440:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6337.888669] Lustre: server umount lustre-MDT0000 complete [ 6341.604123] LustreError: 159431:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6341.622866] LustreError: 159431:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6343.376038] LustreError: 159413:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785399580 with bad export cookie 6605417393421094149 [ 6343.414900] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6344.980538] Lustre: server umount lustre-MDT0001 complete [ 6363.663268] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 6368.592735] Key type lgssc unregistered [ 6368.898035] LNet: 163004:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6368.905138] LNetError: 163004:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6368.928953] LNet: Removed LNI 192.168.202.134@tcp [ 6369.882358] Key type .llcrypt unregistered [ 6369.891937] Key type ._llcrypt unregistered [ 6408.099060] Key type ._llcrypt registered [ 6408.100296] Key type .llcrypt registered [ 6408.358995] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 6409.716555] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6409.773043] alg: No test for adler32 (adler32-zlib) [ 6411.391073] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [ 6411.697229] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 6413.337832] Key type lgssc registered [ 6414.470498] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6435.469169] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 04:21:09 (1785399669) [ 6460.138374] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [ 6477.799586] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 6536.869514] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6536.900503] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6538.283564] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6538.350477] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6538.604604] Lustre: lustre-MDT0000: new disk, initializing [ 6538.719221] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6538.738385] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6544.264474] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6558.091284] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6558.300842] Lustre: 167307:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6558.431783] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6558.436039] Lustre: Skipped 1 previous similar message [ 6558.605164] Lustre: lustre-MDT0001: new disk, initializing [ 6558.803537] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6558.842960] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6558.856526] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6565.254952] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6578.135838] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6586.149827] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6596.681374] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6597.115434] Lustre: lustre-OST0000: new disk, initializing [ 6597.122071] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6597.132770] Lustre: 169278:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6597.273584] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6602.913288] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6602.933479] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6603.047923] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6605.945904] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6618.165744] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6633.957483] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6633.965147] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6633.985467] Lustre: Skipped 1 previous similar message [ 6633.991412] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6639.299525] Lustre: server umount lustre-OST0000 complete [ 6645.729982] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6645.746269] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6645.766171] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6645.780762] Lustre: Skipped 1 previous similar message [ 6649.314274] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6649.319289] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6649.357848] Lustre: Skipped 1 previous similar message [ 6652.605537] Lustre: server umount lustre-MDT0000 complete [ 6654.434683] LustreError: 167318:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6654.464226] LustreError: 167318:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 6658.379728] LustreError: 169277:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785399895 with bad export cookie 15837406279668498943 [ 6658.383709] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6658.393223] LustreError: 169277:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 6659.244286] Lustre: server umount lustre-MDT0001 complete [ 6680.690622] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 6686.012897] Key type lgssc unregistered [ 6686.534241] LNet: 171204:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6686.560139] LNetError: 171204:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6686.580670] LNet: Removed LNI 192.168.202.134@tcp [ 6688.599807] Key type .llcrypt unregistered [ 6688.603046] Key type ._llcrypt unregistered [ 6724.442922] Key type ._llcrypt registered [ 6724.444656] Key type .llcrypt registered [ 6724.609838] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 6725.868252] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6725.915169] alg: No test for adler32 (adler32-zlib) [ 6727.230653] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [ 6727.569645] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 6729.344516] Key type lgssc registered [ 6730.840790] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6740.952459] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6740.984574] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6742.707921] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6748.216540] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6753.251298] LustreError: 172628:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6753.283406] LustreError: 172628:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6758.370386] LustreError: 172629:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6762.164500] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6763.098810] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6768.773367] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6778.368642] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6786.387769] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6797.965958] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6798.977255] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6806.486818] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 6814.366733] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 6822.276432] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6832.536513] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6832.580993] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6832.606448] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6832.618734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6832.638659] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6833.853902] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6833.898497] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6833.931602] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6833.950356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6833.972576] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6835.400417] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6835.427848] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6835.450508] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6835.458704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6835.483436] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6836.618895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6837.745171] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6837.753847] Lustre: Skipped 1 previous similar message [ 6837.775165] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6837.792859] Lustre: Skipped 1 previous similar message [ 6837.801875] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6837.810101] Lustre: Skipped 1 previous similar message [ 6837.815331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6837.826100] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6837.833068] Lustre: Skipped 1 previous similar message [ 6838.902582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6840.012558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6840.857797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6841.739597] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6841.756562] Lustre: Skipped 3 previous similar messages [ 6841.780320] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6841.795696] Lustre: Skipped 3 previous similar messages [ 6841.802749] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6841.808865] Lustre: Skipped 3 previous similar messages [ 6841.814410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6843.065235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6843.086684] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6843.102526] Lustre: Skipped 4 previous similar messages [ 6844.229254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6845.333802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6846.562044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6847.692969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6848.647582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6849.538196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6850.701342] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6850.711539] Lustre: Skipped 7 previous similar messages [ 6850.733225] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6850.752114] Lustre: Skipped 7 previous similar messages [ 6850.756459] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6850.759575] Lustre: Skipped 7 previous similar messages [ 6850.762156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6851.827627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6851.846732] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6851.849121] Lustre: Skipped 7 previous similar messages [ 6852.853977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6854.030315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6855.204872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6856.377640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6857.615127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6858.753584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6860.100953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6861.202127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6862.318492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6863.506052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6864.855667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6866.241393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6867.520579] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6867.534934] Lustre: Skipped 13 previous similar messages [ 6867.582892] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6867.599480] Lustre: Skipped 13 previous similar messages [ 6867.603965] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6867.606637] Lustre: Skipped 13 previous similar messages [ 6867.645808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6868.783067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6868.814744] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6868.825941] Lustre: Skipped 13 previous similar messages [ 6870.116442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6871.214213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6872.327084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6873.515109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6874.396782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6875.262509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6876.314926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6877.616257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6878.732720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6879.937430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6881.347296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6882.753090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6884.247076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6886.063092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6887.595022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6888.943223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6889.863036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6890.838178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6891.574367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6892.540875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6893.525555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6894.619908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6895.967187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6897.386824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6898.612350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6900.288041] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6900.295457] Lustre: Skipped 26 previous similar messages [ 6900.343893] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6900.360503] Lustre: Skipped 26 previous similar messages [ 6900.384420] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6900.391608] Lustre: Skipped 26 previous similar messages [ 6900.396944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6902.044738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6902.060542] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6902.076246] Lustre: Skipped 26 previous similar messages [ 6903.690096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6905.124557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6906.783437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6908.488855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6910.461338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6912.068704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6913.340805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6914.633972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6915.987311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6917.336562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6918.917689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6920.290389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6922.489516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6924.309831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6925.407340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6926.159722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6927.283574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6928.339495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6929.423450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6930.696939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6932.161069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6934.030604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6935.291526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6936.689371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6937.937371] LustreError: 172648:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6937.958506] LustreError: 172648:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6937.997202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6939.246243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6940.482972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6941.976927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6943.212910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6944.347879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6945.469172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6946.823156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6948.089341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6949.483172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6950.665570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6951.947091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6952.917805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6954.128418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6955.055914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6956.132673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6957.055274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6958.184386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6959.408836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6960.519542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6961.626544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6963.043812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6964.293102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6965.519418] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6965.536424] Lustre: Skipped 48 previous similar messages [ 6965.549565] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6965.569269] Lustre: Skipped 48 previous similar messages [ 6965.580663] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6965.588222] Lustre: Skipped 48 previous similar messages [ 6965.592927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6966.816840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6966.833735] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6966.842605] Lustre: Skipped 48 previous similar messages [ 6968.020606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6969.394382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6970.715248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6972.374874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6974.423239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6975.935506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6977.553487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6979.164906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6980.939409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6982.712796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6984.333248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6985.765689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6987.850113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6990.119324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6991.738226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6993.491511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6995.383061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6997.180615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6998.838522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7000.858723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7002.319313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7004.055651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7005.725085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7006.998432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7008.374685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7010.032628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7011.040467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7012.019157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7012.962120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7014.422029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7016.037845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7017.466375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7019.061107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7021.003285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7022.362564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7023.700406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7025.012015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7026.910157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7028.175113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7029.965528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7032.005059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7034.520529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7036.119090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7037.455305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7039.256613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7041.379397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7043.480597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7045.964235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7049.708736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7052.759682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7054.183765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7055.722588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7056.562382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7058.281467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7059.982319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7061.565525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7062.998986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7064.238106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7065.816134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7067.652333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7069.382117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7071.013239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7072.608403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7073.830962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7075.225957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7076.490728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7077.654757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7078.827214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7079.760840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7080.933500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7081.845252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7082.893280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7083.860190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7085.964800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7087.014631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7088.131794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7089.411096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7090.723615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7091.881827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7093.414526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7094.653981] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7094.667568] Lustre: Skipped 81 previous similar messages [ 7094.688310] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7094.701148] Lustre: Skipped 81 previous similar messages [ 7094.712841] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7094.716958] Lustre: Skipped 81 previous similar messages [ 7094.736864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7096.327729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7096.346197] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7096.356152] Lustre: Skipped 81 previous similar messages [ 7098.076326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7099.703746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7101.489911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7103.109695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7104.774887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7106.942864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7108.982033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7110.970523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7112.737382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7114.647756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7116.490063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7118.743203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7119.955232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7121.483424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7122.428422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7123.831966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7125.097636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7126.431700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7127.453913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7128.042806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7129.068778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7129.960479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7130.916203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7131.844217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7133.146072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7134.352215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7135.743538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7137.160322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7138.261268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7139.484778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7140.681586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7141.951359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7143.301300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7144.760372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7146.195571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7147.737443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7149.095889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7150.770858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7151.950854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7153.695063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7156.443057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7157.896360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7158.888742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7160.773358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7162.282136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7164.718220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7166.714350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7168.612871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7170.110181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7171.711479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7172.784480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7174.835917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7176.296996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7177.529450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7179.228772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7180.811235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7182.630130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7184.129672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7185.292959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7186.981159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7188.262362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7189.351628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7190.043604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7190.996466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7191.976209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7193.158426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7194.403671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7195.614019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7196.536861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7197.523429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7198.739620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7200.065815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7201.037654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7201.782466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7202.762695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7203.745467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7204.741054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7205.729804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7206.838910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7208.156384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7209.289328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7210.429606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7211.898963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7212.837520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7213.998374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7215.190295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7216.195584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7217.264415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7218.725609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7220.019477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7221.402353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7222.327591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7223.382068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7224.442890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7225.825786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7227.325048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7228.212558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7229.439616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7230.571342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7231.768364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7233.078201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7234.002418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7234.853897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7236.175247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7237.344307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7238.281290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7239.796433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7241.061915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7242.184596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7243.506214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7244.653967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7245.553608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7246.940497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7247.952967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7249.085929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7250.301084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7251.298500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7252.520291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7253.494301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7254.541658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7255.678212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7256.847049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7257.879569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7258.880052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7259.742342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7260.716455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7261.837489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7262.886987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7264.161621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7265.664516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7266.928936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7268.275441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7269.509197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7270.451214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7271.769746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7273.391898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7274.616241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7276.050522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7277.659236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7278.905463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7280.004427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7281.304508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7282.322877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7283.474150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7284.568653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7285.937418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7287.366886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7288.533974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7289.966484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7291.350058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7292.747571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7293.975302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7295.080189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7296.196502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7297.255458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7298.206093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7299.764585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7300.982582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7302.637380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7303.783799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7304.881497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7306.093380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7307.415485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7308.741410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7309.934776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7310.900214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7312.009409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7313.299149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7314.478501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7315.823536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7317.202547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7318.418326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7319.535640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7320.649571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7321.942743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7322.877672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7323.772631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7324.534150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7325.391311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7326.327320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7327.503591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7328.562172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7329.848422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7330.826547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7331.715079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7332.972243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7334.427631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7336.016258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7337.299202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7338.503492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7339.934981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7341.106428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7342.767182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7344.306467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7345.718900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7347.400499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7348.801669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7350.020142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7351.525205] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7351.529101] Lustre: Skipped 199 previous similar messages [ 7351.547405] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7351.561416] Lustre: Skipped 199 previous similar messages [ 7351.564975] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7351.572825] Lustre: Skipped 199 previous similar messages [ 7351.583985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7352.935760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7352.950139] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7352.959512] Lustre: Skipped 199 previous similar messages [ 7354.401030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7356.370958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7358.226710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7359.429534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7360.978128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7362.373561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7363.673602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7365.014784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7366.711662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7368.412743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7369.430481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7370.603518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7371.764584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7373.092280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7374.496578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7376.080605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7377.350045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7378.779136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7379.743642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7380.867939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7382.038295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7383.356658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7384.770372] LustreError: 172648:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7384.781465] LustreError: 172648:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7384.801707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7385.997047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7387.093637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7388.613310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7390.050808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7391.382189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7392.774978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7393.917620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7394.836350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7395.999033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7397.228601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7398.666697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7399.821907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7401.369670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7402.634355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7403.970190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7405.447699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7406.342682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7407.737499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7408.837722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7410.008935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7411.344516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7412.761962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7413.876869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7415.033755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7416.450069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7417.719028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7419.511543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7420.805965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7421.868873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7423.474185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7424.960315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7426.198624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7428.035612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7429.504911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7430.963735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7432.972551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7433.870293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7435.070919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7436.151778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7437.670338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7439.135658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7440.533589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7441.979245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7443.785212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7445.225269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7446.382501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7447.497448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7448.862826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7450.114568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7451.206421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7452.465710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7453.520469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7455.097564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7456.594890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7457.629083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7458.900840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7460.198882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7461.447142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7463.176258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7465.056670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7466.713182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7467.791033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7469.201122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7470.343883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7471.473549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7472.701285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7474.002721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7475.264141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7476.668490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7477.934654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7479.287112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7480.522734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7481.803835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7483.285691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7484.617445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7485.876210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7486.974056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7488.561322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7490.058508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7491.873836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7493.481693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7494.854233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7496.420291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7497.885437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7499.145309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7500.613952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7501.824074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7502.898175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7504.249555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7505.414296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7506.618075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7507.817674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7508.988180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7510.115924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7511.344122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7512.651471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7513.844169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7515.044523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7516.173392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7517.314993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7518.294769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7519.372933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7520.377854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7521.322730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7522.365928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7523.493136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7524.718181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7525.824148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7527.007075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7528.006720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7529.271852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7530.538342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7531.872239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7533.022414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7534.178852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7535.298153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7536.757633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7537.992468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7539.218610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7540.542899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7541.640199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7542.601916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7543.587464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7544.802642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7545.880938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7547.140739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7548.222638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7549.010944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7550.023993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7550.916961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7551.935988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7553.184099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7554.459363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7555.645302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7556.628792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7557.894685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7559.192401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7560.501209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7561.698740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7562.912836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7563.878970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7565.279353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7566.640045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7568.443217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7569.911855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7571.306451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7572.556576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7574.098360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7575.484442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7576.962468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7578.438546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7579.908157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7581.357148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7582.611722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7584.220929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7585.876705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7586.956327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7588.060068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7589.209192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7590.340983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7591.373297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7592.437752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7593.395207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7594.387788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7595.633492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7596.813413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7598.397312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7599.933769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7601.397665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7602.922698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7604.037104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7605.123146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7606.723820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7608.351190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7609.963503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7611.590158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7613.136365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7614.034875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7615.200959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7616.359163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7617.468741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7618.804619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7620.125775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7621.431209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7622.726959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7623.897161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7624.990315] LustreError: 172648:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7625.009281] LustreError: 172648:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7625.015846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7626.225694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7627.502841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7628.574005] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7629.941246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7631.233718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7632.527731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7633.883789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7635.009303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7636.137913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7637.279198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7638.394718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7639.634867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7640.880592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7642.027441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7643.241929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7644.778498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7645.819195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7647.136295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7648.197853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7649.240514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7650.458469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7651.584062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7652.928480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7654.188406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7655.706812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7656.883468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7658.250428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7659.792399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7661.394349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7662.986572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7664.436802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7665.853774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7667.791348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7669.163557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7670.785936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7672.893293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7674.715747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7675.879119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7676.818379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7677.851447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7678.722677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7680.057187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7681.060132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7682.287937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7683.715439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7684.955911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7686.152165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7687.568863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7688.896146] LustreError: 172648:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7688.919459] LustreError: 172648:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7688.937982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7690.149486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7691.433406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7692.896658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7694.083552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7695.231338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7696.379151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7697.588171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7698.697043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7699.912884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7701.154594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7702.208543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7703.470827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7704.617028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7705.617273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7706.806254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7708.764912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7710.409751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7712.794950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7716.103688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7718.029907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7720.557614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7723.370439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7725.597135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7726.916340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7728.428522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7729.287621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7730.606864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7731.706586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7733.531347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7734.509884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7735.767977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7736.986737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7738.668690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7740.024048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7741.499947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7742.807622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7743.921680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7745.242494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7746.683869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7748.705988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7750.801636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7752.804710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7754.616146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7756.720058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7758.417764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7760.000146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7761.425944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7762.724109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7764.269936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7765.723978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7767.634096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7769.010531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7770.160326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7772.658757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7774.809884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7777.682594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7779.632053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7781.921642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7784.048750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7786.375211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7788.441026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7790.908633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7792.351568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7793.412235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7794.530059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7795.704256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7797.072536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7798.299564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7799.645518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7800.956279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7802.243145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7803.660263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7804.685617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7805.925410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7807.306113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7808.615506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7810.247197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7811.691165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7813.103273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7814.364481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7815.452291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7817.009407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7818.442892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7819.721892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7821.143144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7822.283964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7823.341781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7824.192527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7825.093482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7825.926290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7826.879796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7828.487780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7830.034248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7831.080691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7832.484768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7833.820208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7835.057729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7836.937043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7838.722668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7840.302221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7842.835116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7844.722300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7847.034455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7848.329981] LustreError: 172648:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7848.337502] LustreError: 172648:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7848.362819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7850.043915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7851.851877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7854.468110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7856.712568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7858.324034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7859.415930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7860.846418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7862.346138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7863.440075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7865.101967] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7865.116887] Lustre: Skipped 373 previous similar messages [ 7865.151219] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7865.165116] Lustre: Skipped 373 previous similar messages [ 7865.177104] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7865.185773] Lustre: Skipped 373 previous similar messages [ 7865.191882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7865.210725] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7865.221785] Lustre: Skipped 372 previous similar messages [ 7866.501387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7868.087916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7869.363068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7870.318891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7871.244876] LustreError: 172648:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7871.263727] LustreError: 172648:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7871.271901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7872.479716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7873.687580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7874.779295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7875.964863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7877.170868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7878.171462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7879.837273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7881.552616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7883.088944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7883.877582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7885.389324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7886.656291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7887.846662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7888.801827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7890.430823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7891.406442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7893.007197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7894.349217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7895.735322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7897.194900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7898.558616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7899.913181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7901.611535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7902.802478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7904.015755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7905.284818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7906.626452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7907.893380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7909.387529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7910.970704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7912.472716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7914.170170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7915.908171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7917.452139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7918.782377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7920.106952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7921.466934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7922.735842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7924.046856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7925.381944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7926.657928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7927.923394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7929.274097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7930.563807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7931.823218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7933.081816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7934.475908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7936.049524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7937.358207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7938.808362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7940.203009] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7941.473829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7942.998738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7944.122773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7945.650779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7946.896226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7948.000448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7949.279253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7950.402380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7951.609606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7952.915331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7954.638296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7955.711475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7957.038726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7958.826912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7960.531558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7962.257059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7963.424378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7964.975797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7966.337271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7968.079648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7969.710484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7971.275906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7973.064531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7974.848350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7976.327169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7977.722236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7979.013763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7980.759138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7982.231029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7983.602678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7984.876970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7986.308095] LustreError: 172648:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7986.325794] LustreError: 172648:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7986.345229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7987.860896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7989.408695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7990.840729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7992.467643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7993.976734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7995.661228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7997.456891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7999.143565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8000.717621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8002.277456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8003.606108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8004.824301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8006.174512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8007.528947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8009.062420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8010.322897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8011.863820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8013.272555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8015.151631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8016.672125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8018.347253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8019.808020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8021.821263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8023.379752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8025.131629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8027.092862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8029.296500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8031.495203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8033.444800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8035.855701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8038.173922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8039.679459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8040.904456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8042.222850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8043.581315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8044.943195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8046.399276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8047.844912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8049.481301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8050.878836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8052.108360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8053.584831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8055.185253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8056.476839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8057.968402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8059.439163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8061.025217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8063.127241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8065.370411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8067.700740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8069.710683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8071.706538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8073.904360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8075.842650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8076.675473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8077.609364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8078.646679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8079.865843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8081.416332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8082.986621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8084.650329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8085.554724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8086.830268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8088.204996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8089.343803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8090.593697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8092.269412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8093.604100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8094.881717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8096.240154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8097.743207] LustreError: 172648:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 8097.762507] LustreError: 172648:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 8097.778530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8098.971942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8100.401277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8101.881475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8103.204705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8104.548507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8105.752448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8106.865199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8108.314257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8109.678811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8110.821572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8112.170150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8113.620913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8114.638591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8116.061922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8117.443832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8118.721733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8119.835890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8120.999632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8122.508603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8124.032722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8125.624130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8127.218089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8129.113341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8130.887950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8132.236167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8133.745872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8135.331200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8136.642118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8138.140795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8139.607778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8141.056934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8142.311642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8143.523778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8145.284043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8146.480777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8147.919319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8149.304365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8150.495723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8151.629352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8152.634116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8154.111432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8155.672257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8156.821802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8158.517572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8159.932546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8161.386197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8162.760824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8164.211873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8165.477707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8166.778728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8168.143063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8169.389239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8170.587490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8171.865469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8173.191355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8174.324553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8175.252652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8176.528627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8177.830952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8179.031419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8180.218635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8181.568313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8182.933615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8184.130933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8185.039845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8186.049398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8187.245688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8188.630631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8189.848435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8191.238174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8192.480616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8193.918455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8195.220319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8196.591474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8197.833242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8199.241765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8200.575710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8201.944571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8778.770183] LustreError: 176771:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 8778.791717] LustreError: 172648:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 8778.814732] LustreError: 172648:0:(osp_precreate.c:655:osp_precreate_send()) Skipped 1 previous similar message [ 9604.580688] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9604.602132] Lustre: Skipped 237 previous similar messages [ 9604.619506] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9609.706842] LustreError: 226754:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9609.730617] LustreError: 226754:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9609.821313] Lustre: server umount lustre-OST0000 complete [ 9619.948524] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9619.948892] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9619.995096] Lustre: Skipped 2 previous similar messages [ 9622.196266] Lustre: server umount lustre-MDT0000 complete [ 9625.068768] LustreError: 173356:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9625.101823] LustreError: 173356:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9628.095324] LustreError: 173364:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785402864 with bad export cookie 9812050200581596840 [ 9628.113484] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9629.835638] Lustre: server umount lustre-MDT0001 complete [ 9651.962184] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 9660.248151] Key type lgssc unregistered [ 9661.207075] LNet: 228036:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9661.228818] LNetError: 228036:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9661.277616] LNet: Removed LNI 192.168.202.134@tcp [ 9664.240942] Key type .llcrypt unregistered [ 9664.245894] Key type ._llcrypt unregistered [ 9699.872361] Key type ._llcrypt registered [ 9699.876311] Key type .llcrypt registered [ 9700.136180] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 9702.398971] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9702.426845] alg: No test for adler32 (adler32-zlib) [ 9703.648225] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [ 9704.056511] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 9705.840202] Key type lgssc registered [ 9707.225073] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9718.974092] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9719.009756] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9720.579343] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9726.568909] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 9731.042920] LustreError: 229452:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9731.070870] LustreError: 229452:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9736.179319] LustreError: 229453:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9738.482251] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9739.230265] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9747.607353] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 9761.978059] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9771.180341] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9781.868958] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9782.297110] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9789.877234] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 9790.538368] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 9799.876483] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9806.540530] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 9808.719324] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9808.750527] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 9808.766928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 9808.789492] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 9808.790124] LustreError: 229472:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 9813.132210] Lustre: lustre-OST0000: Not available for connect from 192.168.202.34@tcp (stopping) [ 9813.988482] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9813.999733] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9814.030967] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9816.034694] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9816.041910] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9816.052038] Lustre: Skipped 1 previous similar message [ 9817.773155] Lustre: server umount lustre-OST0000 complete [ 9828.835211] LustreError: 229472:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 9854.697778] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9855.330078] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9856.378268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 9856.384364] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 9864.401185] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [ 9865.577368] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 9865.598692] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 9873.565924] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9880.849375] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 4 sec [ 9886.814030] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9f5303b63800.ost_server_uuid 50 [ 9888.709109] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9f5303b63800.ost_server_uuid in FULL state after 0 sec [ 9896.427040] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9896.465083] Lustre: Skipped 1 previous similar message [ 9896.486257] LustreError: 232667:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9896.923408] Lustre: server umount lustre-OST0000 complete [ 9906.682299] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9906.687884] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9906.710270] Lustre: Skipped 1 previous similar message [ 9906.740333] Lustre: Skipped 1 previous similar message [ 9910.814719] Lustre: server umount lustre-MDT0000 complete [ 9911.777972] LustreError: 230181:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9911.825111] LustreError: 230181:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9916.493952] LustreError: 229433:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785403153 with bad export cookie 11357471706471516753 [ 9916.517847] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9917.573936] Lustre: server umount lustre-MDT0001 complete [ 9932.353280] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [ 9936.181305] Key type lgssc unregistered [ 9936.615894] LNet: 234668:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9936.654070] LNetError: 234668:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9936.697857] LNet: Removed LNI 192.168.202.134@tcp [ 9938.581185] Key type .llcrypt unregistered [ 9938.588128] Key type ._llcrypt unregistered [ 9976.695337] Key type ._llcrypt registered [ 9976.697414] Key type .llcrypt registered [ 9976.828889] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [ 9978.040473] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9978.080826] alg: No test for adler32 (adler32-zlib) [ 9979.605483] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [ 9979.898425] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [ 9981.632911] Key type lgssc registered [ 9983.782173] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9997.774981] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9997.821929] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9999.688364] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10005.307163] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10010.090153] LustreError: 236399:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10010.111393] LustreError: 236399:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10015.213755] LustreError: 236400:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10018.024766] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10018.904824] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10026.639137] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10031.609757] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10040.937604] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10041.444109] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10044.541585] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [10051.098698] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10063.991681] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10064.348502] Lustre: lustre-OST0001: new disk, initializing [10064.359284] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10064.368285] Lustre: 238709:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10064.531849] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10069.771319] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10069.791758] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10070.058497] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10073.196720] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10095.664353] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10104.982063] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10131.425430] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10131.436392] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10131.445282] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10131.463142] Lustre: Skipped 2 previous similar messages [10131.488803] Lustre: Skipped 3 previous similar messages [10134.472522] Lustre: server umount lustre-MDT0000 complete [10136.571632] LustreError: 236394:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10136.611724] LustreError: 236394:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10140.754514] LustreError: 239501:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785403377 with bad export cookie 4703244262024605740 [10140.769062] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10140.791923] LustreError: 239501:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10141.768489] Lustre: server umount lustre-MDT0001 complete [10157.792275] Lustre: 235661:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785403378/real 1785403378] req@ffff916e3bd8a680 x1872130962727296/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785403394 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [10157.860676] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10157.904562] Lustre: Skipped 1 previous similar message [10159.403521] Lustre: server umount lustre-OST0000 complete [10161.833522] Lustre: 235663:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785403382/real 1785403382] req@ffff916d226e5c00 x1872130962727552/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785403398 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [10163.040592] Lustre: 235661:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785403383/real 1785403383] req@ffff916d226e6680 x1872130962727808/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785403399 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [10166.184589] Lustre: server umount lustre-OST0001 complete [10195.771666] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [10209.892384] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10210.463517] LustreError: 241842:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10210.548703] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10215.906914] LustreError: 241843:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10216.671940] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10226.145687] LustreError: 241842:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10226.164195] LustreError: 241842:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10227.362258] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10228.066552] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10234.092907] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10238.292149] Lustre: 242985:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10247.335849] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10247.915103] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10253.045323] LustreError: 243340:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10255.855284] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10256.129213] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [10269.757701] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10280.116629] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10289.738954] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10294.700774] Lustre: 244857:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10310.117111] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10310.130127] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10310.158695] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [10311.179199] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [10311.188255] Lustre: Skipped 3 previous similar messages [10315.731908] Lustre: server umount mylustre-MDT0000 complete [10316.266055] LustreError: 241843:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10316.297751] LustreError: 241843:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [10320.865246] LustreError: 241818:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785403557 with bad export cookie 4703244262024608512 [10320.879186] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10320.891569] LustreError: 241818:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [10321.379595] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10321.384637] Lustre: Skipped 4 previous similar messages [10321.699175] Lustre: server umount mylustre-MDT0001 complete [10328.095265] Lustre: server umount mylustre-OST0000 complete [10335.244806] Lustre: server umount mylustre-OST0001 complete [10367.213867] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [10382.124759] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10382.576090] LustreError: 247169:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10382.593643] LustreError: 247169:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [10382.654068] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10382.661028] Lustre: Skipped 1 previous similar message [10389.076289] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10401.998621] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10409.809048] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10414.259440] Lustre: 248313:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10424.656273] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10427.432366] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [10434.988547] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10448.886110] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10449.636745] Lustre: tfs-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10449.645032] Lustre: Skipped 2 previous similar messages [10459.080361] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10471.099174] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10476.299052] Lustre: 250187:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10489.832523] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10489.845534] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10489.866619] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [10494.054418] Lustre: server umount tfs-MDT0000 complete [10498.664810] LustreError: 247146:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785403735 with bad export cookie 4703244262024610178 [10498.669626] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10498.680890] LustreError: 247146:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10499.299927] Lustre: server umount tfs-MDT0001 complete [10514.040922] Lustre: server umount tfs-OST0000 complete [10516.463610] Lustre: 235661:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785403737/real 1785403737] req@ffff916e05674a80 x1872130962859520/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785403753 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [10516.505954] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10516.521994] Lustre: Skipped 3 previous similar messages [10520.113631] Lustre: server umount tfs-OST0001 complete [10549.539374] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [10573.272471] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10573.572715] LustreError: 252443:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10573.582815] LustreError: 252443:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 17 previous similar messages [10578.896119] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10592.340498] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10593.519292] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10593.522015] Lustre: Skipped 1 previous similar message [10601.209506] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10605.824439] Lustre: 253590:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10615.151214] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10618.963797] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [10626.086578] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10639.829395] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10652.361568] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10664.197691] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10670.351343] Lustre: 255461:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10683.049341] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 05:31:56 (1785403916) [10705.890581] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10705.895118] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10705.904206] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10705.908340] Lustre: Skipped 4 previous similar messages [10709.775272] Lustre: server umount lustre-MDT0000 complete [10715.145530] LustreError: 252420:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785403951 with bad export cookie 4703244262024611620 [10715.169498] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10717.182401] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10717.199756] Lustre: Skipped 5 previous similar messages [10722.498194] Lustre: server umount lustre-MDT0001 complete [10729.188749] Lustre: server umount lustre-OST0000 complete [10737.623600] Lustre: server umount lustre-OST0001 complete [10746.780386] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [10760.435492] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [10808.999247] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [10808.999247] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [10808.999247] [10809.027382] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [10809.404261] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10809.459963] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10809.573277] Lustre: lustre-MDT0000: new disk, initializing [10809.803456] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10816.904299] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10830.447456] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [10830.447456] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [10830.447456] [10830.480532] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [10830.674231] Lustre: 258651:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10830.795559] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10830.801492] Lustre: Skipped 1 previous similar message [10830.976095] Lustre: lustre-MDT0001: new disk, initializing [10831.260878] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10831.290387] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10839.805964] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10864.431540] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10865.215679] Lustre: lustre-OST0000: new disk, initializing [10865.218060] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10865.250467] Lustre: 260235:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10865.467823] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10865.470789] Lustre: Skipped 4 previous similar messages [10867.001239] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10867.027580] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10867.228663] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10876.812200] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10891.904861] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10892.193957] Lustre: lustre-OST0001: new disk, initializing [10892.214811] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10892.226732] Lustre: 261110:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10893.993705] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10894.030856] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10894.183092] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10899.544539] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [10913.566159] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 05:35:47 (1785404147) [10942.223579] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 05:36:16 (1785404176) [10959.333565] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10959.339581] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10959.349102] LustreError: Skipped 1 previous similar message [10959.352082] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10959.366166] Lustre: Skipped 6 previous similar messages [10959.393380] Lustre: Skipped 3 previous similar messages [10973.152129] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10973.490578] Lustre: server umount lustre-OST0000 complete [10976.259496] LustreError: 260296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10976.285162] LustreError: 260296:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 23 previous similar messages [10979.809403] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10986.301534] Lustre: server umount lustre-MDT0000 complete [10991.934787] LustreError: 258642:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785404228 with bad export cookie 4703244262024612950 [10991.948220] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10991.975775] LustreError: 258642:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [10996.709658] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10996.719196] Lustre: Skipped 10 previous similar messages [10998.996859] Lustre: server umount lustre-MDT0001 complete [11015.944277] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [11031.809650] Lustre: server umount lustre-OST0001 complete [11043.694172] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [11064.689028] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [11111.602087] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11112.259688] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11112.368089] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11112.686063] Lustre: lustre-MDT0000: new disk, initializing [11112.895968] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11120.960903] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [11135.615653] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11135.740865] Lustre: 265702:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11135.793456] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11135.796325] Lustre: Skipped 1 previous similar message [11135.845249] Lustre: lustre-MDT0001: new disk, initializing [11135.965695] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11135.974935] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11142.971429] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [11154.754417] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11163.431740] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11179.487298] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11180.018888] Lustre: lustre-OST0000: new disk, initializing [11180.026743] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11180.041497] Lustre: 267673:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11182.016666] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11182.041245] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11182.365290] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11187.964384] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [11199.493593] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11212.771052] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11212.784838] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11212.804433] Lustre: Skipped 4 previous similar messages [11212.817389] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11216.299609] Lustre: server umount lustre-OST0000 complete [11230.933756] Lustre: server umount lustre-MDT0000 complete [11237.755341] LustreError: 265696:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785404474 with bad export cookie 4703244262024614420 [11237.759876] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11237.768548] LustreError: 265696:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11239.709146] Lustre: server umount lustre-MDT0001 complete [11257.220809] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [11261.622129] Key type lgssc unregistered [11262.062647] LNet: 269450:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11262.075776] LNetError: 269450:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11262.092855] LNet: Removed LNI 192.168.202.134@tcp [11263.556749] Key type .llcrypt unregistered [11263.562375] Key type ._llcrypt unregistered [11300.964508] Key type ._llcrypt registered [11300.971874] Key type .llcrypt registered [11301.191805] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [11303.150277] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11303.562624] alg: No test for adler32 (adler32-zlib) [11304.867838] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [11305.125520] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [11306.888303] Key type lgssc registered [11308.876842] Lustre: Echo OBD driver; http://www.lustre.org/ [11322.290827] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11322.340860] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11324.257894] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11333.614880] LustreError: 270920:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11333.661404] LustreError: 270920:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11334.712083] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [11338.750387] LustreError: 270920:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11343.847506] LustreError: 270919:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11348.976931] LustreError: 270920:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11354.099329] LustreError: 270919:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11354.214872] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11354.398490] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11363.271669] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [11378.590518] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11385.977371] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11395.735492] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11396.197472] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11407.040923] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [11418.834614] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [11421.764557] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11423.489441] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [11429.351899] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11429.356098] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11429.391752] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11429.416451] Lustre: Skipped 1 previous similar message [11433.005300] Lustre: server umount lustre-OST0000 complete [11439.586698] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11439.591806] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11439.612016] Lustre: Skipped 2 previous similar messages [11439.634630] Lustre: Skipped 1 previous similar message [11444.705664] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11445.333668] Lustre: server umount lustre-MDT0000 complete [11449.832157] LustreError: 272535:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11449.880867] LustreError: 272535:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11451.573239] LustreError: 272768:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785404688 with bad export cookie 537255715770722481 [11451.582912] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11451.606036] LustreError: 272768:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11452.852839] Lustre: server umount lustre-MDT0001 complete [11473.894950] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [11480.395277] Key type lgssc unregistered [11480.652191] LNet: 274537:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11480.667285] LNetError: 274537:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11480.691462] LNet: Removed LNI 192.168.202.134@tcp [11482.007460] Key type .llcrypt unregistered [11482.013688] Key type ._llcrypt unregistered [11502.567757] Key type ._llcrypt registered [11502.571565] Key type .llcrypt registered [11502.708127] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [11504.179480] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11504.218225] alg: No test for adler32 (adler32-zlib) [11505.585503] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [11505.884966] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [11507.576182] Key type lgssc registered [11509.351919] Lustre: Echo OBD driver; http://www.lustre.org/ [11522.481766] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11522.529040] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11538.211865] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [11543.328516] LustreError: 275735:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.134@tcp: failed processing log, type 4: rc = -110 [11570.016223] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11578.563618] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [11592.087439] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11595.696720] Lustre: server umount lustre-OST0000 complete [11621.787578] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [11627.322938] Key type lgssc unregistered [11628.019546] LNet: 277369:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11628.049445] LNetError: 277369:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11629.094824] LNet: Removed LNI 192.168.202.134@tcp [11630.644731] Key type .llcrypt unregistered [11630.652812] Key type ._llcrypt unregistered [11689.373311] Key type ._llcrypt registered [11689.377539] Key type .llcrypt registered [11689.515612] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [11690.963584] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11691.156509] alg: No test for adler32 (adler32-zlib) [11692.163568] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [11692.442978] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [11694.136195] Key type lgssc registered [11696.049684] Lustre: Echo OBD driver; http://www.lustre.org/ [11714.859067] loop0: detected capacity change from 0 to 204800000 [11715.203301] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11715.239430] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11716.204878] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11717.551351] loop0: detected capacity change from 0 to 204800000 [11717.830405] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11717.847530] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11718.466600] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11720.601886] loop0: detected capacity change from 0 to 204800000 [11721.222124] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11721.282772] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11721.610981] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11723.790865] loop0: detected capacity change from 0 to 204800000 [11724.403478] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11724.426771] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11724.585431] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [11726.256103] loop0: detected capacity change from 0 to 204800000 [11726.315591] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [11732.527307] loop0: detected capacity change from 0 to 204800000 [11732.594329] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [11738.207182] loop0: detected capacity change from 0 to 204800000 [11738.253803] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [11744.813033] loop0: detected capacity change from 0 to 204800000 [11744.879997] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [11751.707616] loop0: detected capacity change from 0 to 204800000 [11751.922875] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11752.055145] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11752.211510] Lustre: 280793:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [11752.235749] Lustre: 280793:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [11752.263085] Lustre: 280793:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [11752.297981] Lustre: 280793:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [11752.325194] Lustre: 280793:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [11752.356933] Lustre: 280793:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [11752.398326] Lustre: 280793:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [11752.422920] Lustre: 280793:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [11752.451242] Lustre: 280793:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [11752.472968] Lustre: 280793:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [11752.495494] Lustre: 280793:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [11752.536107] Lustre: 280793:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [11752.570583] Lustre: 280793:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [11752.598947] Lustre: 280793:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [11761.710992] Lustre: server umount MGS complete [11763.459831] loop0: detected capacity change from 0 to 204800000 [11763.670198] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11764.811413] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11764.945500] Lustre: lustre-MDT0000: Aborting client recovery [11766.738574] loop1: detected capacity change from 0 to 204800000 [11766.965862] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11767.221866] Lustre: 281227:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [11767.246945] Lustre: 281227:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [11767.293868] Lustre: 281227:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [11767.326770] Lustre: 281227:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [11767.357784] Lustre: 281227:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [11767.389921] Lustre: 281227:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [11767.425437] Lustre: 281227:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [11767.478724] Lustre: Found index 1 for lustre-MDT0001, updating log [11767.966082] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11768.081794] Lustre: lustre-MDT0001: Aborting client recovery [11769.420074] loop2: detected capacity change from 0 to 204800000 [11769.806107] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11769.940508] Lustre: 281307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [11769.954598] Lustre: 281307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [11769.968280] Lustre: 281307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [11769.986966] Lustre: 281307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [11770.029261] Lustre: 281307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [11770.051863] Lustre: 281307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [11770.073404] Lustre: 281307:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [11770.244567] Lustre: Found index 0 for lustre-OST0000, updating log [11770.275382] Lustre: 281328:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11770.351409] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11770.373541] Lustre: lustre-OST0000: Aborting recovery [11771.653160] loop3: detected capacity change from 0 to 204800000 [11771.867458] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11771.982159] Lustre: 281394:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [11772.006790] Lustre: 281394:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [11772.047250] Lustre: 281394:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [11772.086455] Lustre: 281394:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [11772.136305] Lustre: 281394:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [11772.162562] Lustre: 281394:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [11772.194071] Lustre: 281394:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [11772.284865] Lustre: Found index 1 for lustre-OST0001, updating log [11772.473160] Lustre: 281404:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11772.582736] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11772.614281] Lustre: lustre-OST0001: Aborting recovery [11772.715657] LustreError: 281170:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -108 [11774.679903] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [11774.684465] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [11774.838913] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [11774.907156] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [11777.770300] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [11777.772103] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [11784.215237] LustreError: 281137:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 2527us: rc = -2 [11784.731051] LustreError: 281135:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 2170us: rc = -2 [11784.772994] LustreError: 281135:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 10 previous similar messages [11785.751399] LustreError: 281134:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 461us: rc = -2 [11785.780214] LustreError: 281134:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 24 previous similar messages [11803.617061] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11803.626583] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11803.626921] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11803.675936] Lustre: Skipped 3 previous similar messages [11805.895985] Lustre: server umount lustre-MDT0000 complete [11807.365442] LustreError: 281120:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785405044 with bad export cookie 3233811165942544028 [11807.379518] LustreError: 281120:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11807.381832] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11808.736621] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11808.738467] LustreError: 281136:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11808.741563] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11808.741570] Lustre: Skipped 3 previous similar messages [11808.750846] Lustre: Skipped 1 previous similar message [11808.796894] LustreError: 281136:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11812.771338] LustreError: 281134:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11812.771636] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11812.800309] Lustre: Skipped 2 previous similar messages [11814.565345] Lustre: server umount lustre-MDT0001 complete [11823.028341] Lustre: server umount lustre-OST0000 complete [11831.072780] Lustre: server umount lustre-OST0001 complete [11844.053547] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 05:51:18 (1785405078) [11883.969541] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [11908.275622] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [11970.450252] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11971.101649] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11971.211720] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11971.462302] Lustre: lustre-MDT0000: new disk, initializing [11971.625839] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11971.645608] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11980.054470] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [11999.937554] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12000.068271] Lustre: 285503:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12000.125907] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12000.132335] Lustre: Skipped 1 previous similar message [12000.241209] Lustre: lustre-MDT0001: new disk, initializing [12000.331796] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12000.379479] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12000.397065] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12008.204798] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12019.431787] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12030.449431] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12041.244894] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12041.615176] Lustre: lustre-OST0000: new disk, initializing [12041.621325] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12041.625678] Lustre: 287478:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12041.737144] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12048.501522] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12048.524539] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12048.696155] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12051.554653] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12062.500932] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12069.489095] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [12085.730507] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [12094.297782] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [12146.150680] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12146.154373] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12146.206530] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12146.224205] Lustre: Skipped 1 previous similar message [12151.269894] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12151.277681] Lustre: Skipped 1 previous similar message [12156.391576] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12156.772927] Lustre: server umount lustre-OST0000 complete [12164.066101] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12164.081455] LustreError: Skipped 1 previous similar message [12164.083069] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12164.108860] Lustre: Skipped 1 previous similar message [12164.118780] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12164.120889] Lustre: Skipped 2 previous similar messages [12169.063983] Lustre: server umount lustre-MDT0000 complete [12171.745961] LustreError: 285514:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12171.780959] LustreError: 285514:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12173.954764] LustreError: 285493:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785405410 with bad export cookie 3233811165942546681 [12173.963299] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12173.971140] LustreError: 285493:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12174.551276] Lustre: server umount lustre-MDT0001 complete [12188.308152] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12196.125691] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12205.825482] Lustre: server umount MGS complete [12223.492572] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12224.286179] LustreError: 291348:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12224.338981] LustreError: 291361:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [12224.520401] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12229.620965] LustreError: 291349:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12230.736254] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12239.852330] LustreError: 291348:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12239.900141] LustreError: 291348:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12244.390699] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12244.966193] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [12244.984449] Lustre: Skipped 2 previous similar messages [12253.734136] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12267.115467] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12276.419901] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12288.904817] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12289.629285] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12289.632079] Lustre: Skipped 1 previous similar message [12300.387058] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12311.419571] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12350.952080] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12350.976129] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12350.993951] Lustre: Skipped 1 previous similar message [12350.997729] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12355.458973] Lustre: server umount lustre-OST0000 complete [12362.209972] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12362.219668] LustreError: 278751:0:(client.c:1381:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000d54c420e x1872132759041408/t0(0) o41->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 224/368 e 0 to 0 dl 0 ref 1 fl Rpc:QU/200/ffffffff rc 0/-1 job:'osp-pre-1-0.0' uid:0 gid:0 projid:4294967295 [12362.247330] Lustre: Skipped 2 previous similar messages [12369.183594] Lustre: server umount lustre-MDT0000 complete [12372.453835] LustreError: 292078:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12372.473859] LustreError: 292078:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12375.657350] LustreError: 291330:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785405612 with bad export cookie 3233811165942548508 [12375.671115] LustreError: 291330:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12375.724534] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12377.343751] Lustre: server umount lustre-MDT0001 complete [12401.345493] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [12405.672295] Key type lgssc unregistered [12406.156563] LNet: 295320:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12406.167159] LNetError: 295320:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12406.196648] LNet: Removed LNI 192.168.202.134@tcp [12407.367421] Key type .llcrypt unregistered [12407.371436] Key type ._llcrypt unregistered [12461.145671] Key type ._llcrypt registered [12461.152723] Key type .llcrypt registered [12461.338929] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [12484.051820] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [12486.864762] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12487.050343] alg: No test for adler32 (adler32-zlib) [12488.338516] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [12488.517532] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [12490.137849] Key type lgssc registered [12491.871191] Lustre: Echo OBD driver; http://www.lustre.org/ [12538.881137] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12538.964724] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12540.489416] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12540.558946] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12540.690373] Lustre: lustre-MDT0000: new disk, initializing [12540.825855] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12540.849752] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12550.558278] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12565.083242] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12565.194941] Lustre: 299295:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12565.241843] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12565.244902] Lustre: Skipped 1 previous similar message [12565.304679] Lustre: lustre-MDT0001: new disk, initializing [12565.393903] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12565.422359] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12565.439560] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12571.434695] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12583.705857] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12591.612987] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12600.572976] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12601.048025] Lustre: lustre-OST0000: new disk, initializing [12601.054180] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12601.070363] Lustre: 301267:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12601.205250] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12610.562232] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12611.205225] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12611.228060] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12611.465576] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12625.594563] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12634.782455] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [12651.817279] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [12660.955912] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [12669.891446] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [12734.438509] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12734.446627] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12734.448145] LustreError: Skipped 1 previous similar message [12734.477220] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12734.482051] Lustre: Skipped 1 previous similar message [12734.545168] Lustre: Skipped 1 previous similar message [12739.561992] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12739.566587] Lustre: Skipped 1 previous similar message [12744.694880] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12744.719586] Lustre: Skipped 1 previous similar message [12746.210681] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12746.971466] Lustre: server umount lustre-OST0000 complete [12754.915177] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12754.917150] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12754.934909] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12754.937872] Lustre: Skipped 1 previous similar message [12758.981953] Lustre: server umount lustre-MDT0000 complete [12764.032540] LustreError: 302520:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785406000 with bad export cookie 3918641783696879062 [12764.044524] LustreError: 302520:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12764.046699] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12764.489704] Lustre: server umount lustre-MDT0001 complete [12776.269829] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12784.798863] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12792.855252] Lustre: server umount MGS complete [12812.187092] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12812.793585] LustreError: 305149:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12812.823081] LustreError: 305149:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12812.835904] LustreError: 305162:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [12812.935846] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12818.424734] LustreError: 305149:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12820.071478] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12822.505487] LustreError: 305150:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12827.629841] LustreError: 305149:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12831.421198] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12832.153677] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12839.422440] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12849.707832] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12858.185647] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12867.544368] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12868.066026] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12877.021600] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [12888.181201] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12923.363642] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12923.377980] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12923.406758] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12923.415848] Lustre: Skipped 1 previous similar message [12928.501531] Lustre: server umount lustre-OST0000 complete [12939.745259] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12939.745728] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12939.767929] Lustre: Skipped 2 previous similar messages [12939.770039] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12939.810067] Lustre: Skipped 3 previous similar messages [12941.383626] Lustre: server umount lustre-MDT0000 complete [12944.866401] LustreError: 305149:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12944.905316] LustreError: 305149:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12945.979854] LustreError: 305130:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785406182 with bad export cookie 3918641783696881001 [12945.999287] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12946.000637] LustreError: 305130:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12946.550114] Lustre: server umount lustre-MDT0001 complete [12965.009575] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [12969.466514] Key type lgssc unregistered [12969.916570] LNet: 309119:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12969.932690] LNetError: 309119:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12969.987186] LNet: Removed LNI 192.168.202.134@tcp [12971.735314] Key type .llcrypt unregistered [12971.737792] Key type ._llcrypt unregistered [13001.489766] Key type ._llcrypt registered [13001.495291] Key type .llcrypt registered [13001.605282] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [13028.174415] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [13030.177515] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13030.273630] alg: No test for adler32 (adler32-zlib) [13031.617325] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [13032.030868] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [13033.984318] Key type lgssc registered [13037.243897] Lustre: Echo OBD driver; http://www.lustre.org/ [13079.661588] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [13104.282241] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13104.348751] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13105.565076] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13105.617678] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13105.831057] Lustre: lustre-MDT0000: new disk, initializing [13106.008365] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13106.052254] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13112.401868] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [13129.307161] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13129.875602] Lustre: lustre-OST0000: new disk, initializing [13129.885468] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13129.887669] Lustre: Skipped 1 previous similar message [13129.915935] Lustre: 313177:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13130.112551] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13135.518836] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [13135.539977] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [13135.606600] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [13138.889643] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [14374.891859] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14374.920023] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14379.825725] Lustre: server umount lustre-MDT0000 complete [14385.231556] LustreError: 312363:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785407621 with bad export cookie 10742428946071917687 [14385.233233] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14385.243810] LustreError: 312363:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [14385.625414] Lustre: server umount lustre-OST0000 complete [14415.749583] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [14434.385740] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [14506.517647] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14507.113387] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14507.214622] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14507.358207] Lustre: lustre-MDT0000: new disk, initializing [14507.554060] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14507.586441] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14514.691497] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [14531.450550] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14531.680322] Lustre: 317601:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14531.759725] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14531.767041] Lustre: Skipped 1 previous similar message [14531.925146] Lustre: lustre-MDT0001: new disk, initializing [14532.129550] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14532.198492] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14532.217147] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14539.148974] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [14550.737931] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14558.780902] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14570.557731] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14571.034281] Lustre: lustre-OST0000: new disk, initializing [14571.038461] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14571.045922] Lustre: 319573:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14571.134853] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14579.300140] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14579.323068] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14579.469991] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [14579.498867] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14592.631318] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14600.246871] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [14600.459660] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [14607.586374] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [14607.972311] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [14610.413801] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [14610.436674] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14610.460172] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [14615.521301] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [14615.532408] Lustre: Skipped 2 previous similar messages [14616.834489] Lustre: server umount lustre-OST0000 complete [14624.226762] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14624.239366] LustreError: Skipped 1 previous similar message [14624.243410] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14624.254058] Lustre: Skipped 1 previous similar message [14624.279575] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14629.101488] Lustre: server umount lustre-MDT0000 complete [14633.502060] LustreError: 319583:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785407870 with bad export cookie 10742428946074858534 [14633.512966] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14633.518741] LustreError: 319583:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14634.586121] Lustre: server umount lustre-MDT0001 complete [14638.870791] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [14641.437737] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 06:37:55 (1785407875) [14652.678986] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14653.190396] LustreError: 321890:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14653.217020] LustreError: 321890:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [14653.284966] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14658.530241] LustreError: 321891:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14659.034761] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [14662.628325] LustreError: 321890:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14667.745613] LustreError: 321891:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14670.524136] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14671.162443] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14677.521175] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [14689.190292] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14695.450259] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14704.071327] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14704.642653] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14712.680357] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [14724.310133] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14745.474698] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14745.893878] Lustre: lustre-OST0001: new disk, initializing [14745.903945] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [14745.919800] Lustre: 324732:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14746.143867] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [14755.538630] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [14755.564610] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [14755.665337] LustreError: 325022:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [14756.713979] LustreError: 325022:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [14756.752517] LustreError: 325022:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [14757.803790] LustreError: 325022:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [14757.835865] LustreError: 325022:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [14758.369908] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [14759.931407] LustreError: 325022:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [14759.961118] LustreError: 325022:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [14764.133676] LustreError: 325028:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [14764.190109] LustreError: 325028:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [14772.836949] LustreError: 325028:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [14772.850653] LustreError: 325028:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 12 previous similar messages [14782.591512] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [14782.878755] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [14789.483534] LustreError: 325028:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [14789.514380] LustreError: 325028:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 29 previous similar messages [14793.750112] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9f53351da800.ost_server_uuid 50 [14796.086899] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9f53351da800.ost_server_uuid in IDLE state after 0 sec [14804.571873] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9f53351da800.ost_server_uuid 50 [14807.338728] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9f53351da800.ost_server_uuid in IDLE state after 0 sec [14810.136075] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [14825.451521] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [14825.473438] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14825.504642] Lustre: Skipped 1 previous similar message [14825.510457] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [14825.530726] Lustre: Skipped 2 previous similar messages [14831.581565] Lustre: server umount lustre-OST0001 complete [14833.645187] LustreError: 323713:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14843.874110] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14843.886357] LustreError: 323713:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14843.900495] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [14843.900503] Lustre: Skipped 5 previous similar messages [14843.903522] Lustre: Skipped 2 previous similar messages [14843.934432] LustreError: 323713:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [14849.121553] Lustre: server umount lustre-OST0000 complete [14856.162129] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14856.181617] LustreError: Skipped 1 previous similar message [14856.207447] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14862.327713] Lustre: server umount lustre-MDT0000 complete [14864.357926] LustreError: 321891:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14864.411610] LustreError: 321891:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [14869.416365] LustreError: 321870:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785408106 with bad export cookie 10742428946074859416 [14869.417786] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14869.442253] LustreError: 321870:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14870.719280] Lustre: server umount lustre-MDT0001 complete [14888.813378] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [14892.930424] Key type lgssc unregistered [14893.442284] LNet: 327439:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14893.470182] LNetError: 327439:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14893.492976] LNet: Removed LNI 192.168.202.134@tcp [14895.133781] Key type .llcrypt unregistered [14895.141344] Key type ._llcrypt unregistered [14931.847593] Key type ._llcrypt registered [14931.850677] Key type .llcrypt registered [14932.097611] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [14933.403501] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [14933.462052] alg: No test for adler32 (adler32-zlib) [14934.647534] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [14935.024925] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [14936.808474] Key type lgssc registered [14938.864548] Lustre: Echo OBD driver; http://www.lustre.org/ [14951.744995] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [14951.769039] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14953.364347] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14959.104493] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [14963.701345] LustreError: 328863:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14963.750796] LustreError: 328863:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [14968.813041] LustreError: 328864:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14973.923561] LustreError: 328863:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14975.027039] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14979.077145] LustreError: 328864:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14984.167032] LustreError: 328863:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14987.428937] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14987.802192] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14995.169857] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [15009.024831] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15009.828526] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [15012.909447] LustreError: 330548:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15019.503693] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [15024.116477] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [15029.232875] LustreError: 330684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15029.249715] LustreError: 330684:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [15030.146337] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [15039.295778] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15039.692894] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [15045.142179] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [15050.059159] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [15060.741123] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [15075.040210] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [15075.615963] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [15198.690805] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [15198.691335] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15198.702074] LustreError: Skipped 1 previous similar message [15198.706680] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [15198.748630] Lustre: Skipped 1 previous similar message [15202.090384] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [15202.105064] Lustre: Skipped 2 previous similar messages [15202.608808] Lustre: server umount lustre-OST0001 complete [15207.427588] LustreError: 330684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15207.472756] LustreError: 330684:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [15217.651534] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15217.669834] Lustre: Skipped 1 previous similar message [15217.670880] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [15217.707289] Lustre: Skipped 1 previous similar message [15220.213506] Lustre: server umount lustre-OST0000 complete [15227.875815] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15227.889484] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [15227.899390] Lustre: Skipped 1 previous similar message [15232.994456] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [15233.600206] Lustre: server umount lustre-MDT0000 complete [15240.119980] LustreError: 328843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785408476 with bad export cookie 9964879742024846904 [15240.160520] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [15240.835193] Lustre: server umount lustre-MDT0001 complete [15260.295711] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [15265.532792] Key type lgssc unregistered [15266.365051] LNet: 333863:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [15266.380317] LNetError: 333863:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [15266.410703] LNet: Removed LNI 192.168.202.134@tcp [15268.143328] Key type .llcrypt unregistered [15268.147406] Key type ._llcrypt unregistered [15333.940764] Key type ._llcrypt registered [15333.944325] Key type .llcrypt registered [15334.161848] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [15336.333246] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [15336.442551] alg: No test for adler32 (adler32-zlib) [15337.975968] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [15338.519814] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [15340.408166] Key type lgssc registered [15342.693721] Lustre: Echo OBD driver; http://www.lustre.org/ [15357.285540] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [15357.301492] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15358.959059] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [15363.592996] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [15369.192082] LustreError: 336172:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15369.212208] LustreError: 336172:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [15374.308177] LustreError: 336173:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15376.924424] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15378.328743] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [15390.454810] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [15397.906202] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [15411.063767] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15411.407823] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [15420.828785] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [15422.640701] LustreError: 337666:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15428.078647] LustreError: 337666:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15429.124891] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [15431.326820] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15431.726878] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [15436.806315] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [15441.560570] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [15453.426232] Lustre: DEBUG MARKER: Using TIMEOUT=20 [15461.773878] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [15482.859801] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15482.883836] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [15482.896463] Lustre: Skipped 3 previous similar messages [15482.915056] Lustre: Skipped 3 previous similar messages [15487.985762] LustreError: 336918:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15488.010390] LustreError: 336918:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [15488.315905] Lustre: server umount lustre-MDT0000 complete [15493.324953] LustreError: 336153:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785408730 with bad export cookie 7876600962379970796 [15493.329825] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [15493.340016] LustreError: 336153:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [15494.077912] Lustre: server umount lustre-MDT0001 complete [15510.119367] Lustre: server umount lustre-OST0000 complete [15513.572307] Lustre: 335433:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785408734/real 1785408734] req@ffff916d093a2300 x1872136582017152/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785408750 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [15513.614156] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15514.943989] Lustre: server umount lustre-OST0001 complete [15539.428306] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [15556.847850] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15557.650143] LustreError: 341449:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15557.690073] LustreError: 341449:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [15557.846030] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [15566.540567] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [15577.582015] LustreError: 341450:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15577.600995] LustreError: 341450:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [15582.671348] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15583.448682] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [15589.590056] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [15593.793651] Lustre: 342591:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [15602.448539] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15603.010172] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [15604.031198] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [15613.438547] LustreError: 342945:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15613.457701] LustreError: 342945:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [15613.592473] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [15628.296201] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15633.990533] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [15643.367996] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [15659.406466] Lustre: DEBUG MARKER: Using TIMEOUT=20 [15665.365943] Lustre: 344466:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [15675.881680] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [15675.884532] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15675.918127] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [15676.974417] Lustre: server umount lustre-MDT0000 complete [15677.946558] LustreError: 341449:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15677.965851] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15677.984901] LustreError: 341449:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [15678.049445] Lustre: Skipped 2 previous similar messages [15685.740868] LustreError: 341429:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785408922 with bad export cookie 7876600962379972497 [15685.779822] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [15687.821285] Lustre: server umount lustre-MDT0001 complete [15703.392422] Lustre: 335435:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785408924/real 1785408924] req@ffff916e1044bb80 x1872136582084608/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785408940 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [15703.428130] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15705.569611] Lustre: 335433:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785408926/real 1785408926] req@ffff916d03106d80 x1872136582084864/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785408942 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [15707.091963] Lustre: server umount lustre-OST0000 complete [15708.720302] Lustre: 335436:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785408929/real 1785408929] req@ffff916d22feca80 x1872136582085120/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785408945 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [15712.924816] Lustre: server umount lustre-OST0001 complete [15717.677079] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [15720.551431] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [15722.711630] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 06:55:57 (1785408957) [15759.255605] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [15775.267653] loop0: detected capacity change from 0 to 1099511627776 [15775.941908] SGI XFS with ACLs, security attributes, quota, no debug enabled [15775.958774] *** VALIDATE xfs *** [15775.985190] XFS (loop0): Mounting V5 Filesystem [15776.059396] XFS (loop0): Ending clean mount [15777.773476] loop1: detected capacity change from 0 to 18691697672192 [15963.302306] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [15976.191218] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [16030.503394] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16030.853771] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [16030.885466] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [16030.983171] Lustre: lustre-MDT0000: new disk, initializing [16031.050392] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [16031.052858] Lustre: Skipped 1 previous similar message [16031.077767] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [16036.555417] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [16050.695650] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16050.924962] Lustre: 349652:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [16051.127312] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [16051.136144] Lustre: Skipped 1 previous similar message [16051.310537] Lustre: lustre-MDT0001: new disk, initializing [16051.636222] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [16051.650334] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [16058.084840] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [16070.044762] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [16077.739573] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [16086.540311] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [16086.992184] Lustre: lustre-OST0000: new disk, initializing [16086.997805] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [16087.020458] Lustre: 351623:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [16088.483523] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [16088.511747] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [16088.679757] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [16096.683669] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [16108.014146] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [16112.780882] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [16112.969959] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [16119.080525] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [16119.490782] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [16124.407659] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [16124.412967] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16124.432733] LustreError: Skipped 1 previous similar message [16124.433672] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [16124.488355] Lustre: Skipped 1 previous similar message [16128.690842] Lustre: server umount lustre-OST0000 complete [16133.601325] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [16133.615971] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [16133.622530] Lustre: Skipped 1 previous similar message [16139.764522] LustreError: 349662:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [16139.794430] LustreError: 349662:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [16140.293353] Lustre: server umount lustre-MDT0000 complete [16145.511332] LustreError: 349644:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785409382 with bad export cookie 7876600962379974051 [16145.533915] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [16145.535993] LustreError: 349644:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [16146.378678] Lustre: server umount lustre-MDT0001 complete [16161.722747] XFS (loop0): Unmounting Filesystem [16171.344333] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 07:03:25 (1785409405) [16182.755858] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16183.654680] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [16183.667464] Lustre: Skipped 2 previous similar messages [16189.793440] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [16202.718521] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16209.650925] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [16219.680186] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [16227.181977] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [16237.927671] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [16247.750307] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [16260.948727] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [16275.439637] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16275.448806] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [16275.452407] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [16275.452414] Lustre: Skipped 2 previous similar messages [16275.463536] Lustre: Skipped 3 previous similar messages [16281.569844] Lustre: server umount lustre-OST0000 complete [16290.276342] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [16290.294878] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [16290.308949] Lustre: Skipped 3 previous similar messages [16294.458712] Lustre: server umount lustre-MDT0000 complete [16299.993095] LustreError: 354921:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785409536 with bad export cookie 7876600962379974933 [16300.011567] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [16300.012612] LustreError: 354921:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [16300.631109] Lustre: server umount lustre-MDT0001 complete [16327.425148] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [16331.496399] Key type lgssc unregistered [16331.849581] LNet: 357847:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [16331.852950] LNetError: 357847:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [16332.904892] LNet: Removed LNI 192.168.202.134@tcp [16334.205245] Key type .llcrypt unregistered [16334.207549] Key type ._llcrypt unregistered [16376.828672] Key type ._llcrypt registered [16376.835254] Key type .llcrypt registered [16377.296253] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [16380.421852] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [16380.487792] alg: No test for adler32 (adler32-zlib) [16381.747269] Lustre: Lustre: Build Version: 2.17.56_2_geb69296 [16382.023951] LNet: Added LNI 192.168.202.134@tcp [8/256/0/180] [16383.721358] Key type lgssc registered [16384.883745] Lustre: Echo OBD driver; http://www.lustre.org/ [16396.020538] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [16396.055885] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16397.760522] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [16404.692962] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [16408.043208] LustreError: 359420:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [16408.099461] LustreError: 359420:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [16413.172740] LustreError: 359421:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [16418.292878] LustreError: 359420:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [16422.372710] LustreError: 359421:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [16425.527119] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16425.968508] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [16438.233091] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [16451.267555] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [16462.141057] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [16475.063069] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [16475.646980] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [16487.261408] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [16499.369376] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [16507.873432] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [16507.892396] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16507.924074] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [16511.457721] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [16511.488228] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16511.514456] Lustre: Skipped 1 previous similar message [16514.105354] Lustre: server umount lustre-MDT0001 complete [16516.579782] LustreError: 361770:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [16526.832957] LustreError: 359415:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [16526.849326] LustreError: 359415:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [16527.175415] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16527.273024] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [16527.275385] Lustre: Found index 1 for lustre-MDT0001, updating log [16527.307049] Lustre: 359409:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [16527.688793] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [16532.981714] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [16533.829923] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [16536.081823] LustreError: 362985:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [16536.095119] LustreError: 362985:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [16536.119428] LustreError: 359413:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [16536.232633] LustreError: 362994:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.134@tcp: cfg command failed: rc = -17 [16536.246409] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [16536.246409] [16547.443969] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.34@tcp (stopping) [16547.457245] Lustre: Skipped 1 previous similar message [16548.327491] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [16548.346713] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16552.542762] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.34@tcp (stopping) [16552.577684] Lustre: Skipped 2 previous similar messages [16553.941447] Lustre: server umount lustre-MDT0001 complete [16557.676741] LustreError: 359416:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.34@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [16557.717621] LustreError: 359416:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [16567.433673] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16567.678654] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [16567.695406] Lustre: Found index 1 for lustre-MDT0001, updating log [16567.804436] Lustre: 359409:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [16568.351260] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [16573.445287] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [16573.477514] LustreError: 363886:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [16573.495558] LustreError: 363886:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [16573.507833] LustreError: 359413:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [16573.515184] LustreError: 359413:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [16573.599990] LustreError: 363891:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [16573.611734] LustreError: 363891:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [16573.624237] LustreError: 363891:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.134@tcp: cfg command failed: rc = -17 [16573.638487] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.134@tcp [16573.638487] [16575.197960] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [16885.382932] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.34@tcp (stopping) [16885.398288] Lustre: Skipped 2 previous similar messages [16885.740034] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16885.757087] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [16885.757438] Lustre: Skipped 1 previous similar message [16889.533793] Lustre: server umount lustre-MDT0001 complete [16890.465776] LustreError: 359415:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.34@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [16890.499188] LustreError: 359415:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [16905.675935] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16905.928252] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [16905.935202] Lustre: Found index 1 for lustre-MDT0001, updating log [16905.994219] Lustre: 359409:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [16906.208789] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [16906.211192] Lustre: Skipped 2 previous similar messages [16906.564703] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [16911.887471] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [16911.912507] LustreError: 371259:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [16911.926585] LustreError: 371259:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [16911.952779] LustreError: 359413:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [16911.955493] LustreError: 359413:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [16912.103879] LustreError: 371264:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [16912.125314] LustreError: 371264:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.134@tcp: cfg command failed: rc = -114 [16912.139084] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [16912.139084] [16914.349804] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [17244.641305] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [17244.648452] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [17244.658101] Lustre: Skipped 2 previous similar messages [17244.679738] Lustre: Skipped 3 previous similar messages [17250.087812] Lustre: server umount lustre-MDT0000 complete [17254.888127] LustreError: 359420:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [17254.937947] LustreError: 359420:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 11 previous similar messages [17255.835600] LustreError: 371065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785410492 with bad export cookie 2361759208757699069 [17255.851356] LustreError: 371065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [17255.863198] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [17260.004341] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [17260.049180] Lustre: Skipped 1 previous similar message [17270.240171] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [17270.826961] Lustre: server umount lustre-MDT0001 complete [17280.244804] Lustre: server umount lustre-OST0000 complete [17292.995119] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [17312.721671] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [17374.524703] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 07:23:28 (1785410608) [17383.407775] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [17383.712645] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [17383.757140] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [17383.919867] Lustre: lustre-MDT0000: new disk, initializing [17384.092725] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [17384.128731] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [17390.781223] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [17404.206968] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [17404.494145] Lustre: 380645:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [17404.630663] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [17404.637940] Lustre: Skipped 1 previous similar message [17404.781866] Lustre: lustre-MDT0001: new disk, initializing [17404.995799] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [17405.083461] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [17405.099208] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [17411.712929] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [17424.360575] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [17431.307924] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [17441.883620] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [17442.485406] Lustre: lustre-OST0000: new disk, initializing [17442.495279] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [17442.508035] Lustre: 382617:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [17442.744744] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [17449.605404] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [17449.620876] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [17449.794087] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [17452.037836] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_default_debug -1 all [17466.854832] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [17480.677713] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [17480.687064] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [17480.701536] Lustre: Skipped 1 previous similar message [17480.714993] Lustre: Skipped 8 previous similar messages [17483.491196] Lustre: server umount lustre-MDT0000 complete [17485.794822] LustreError: 383338:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [17485.825955] LustreError: 383338:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [17488.676868] LustreError: 380638:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785410725 with bad export cookie 2361759208757701029 [17488.687458] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [17488.694895] LustreError: 380638:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [17489.310311] Lustre: server umount lustre-MDT0001 complete [17505.474773] Lustre: server umount lustre-OST0000 complete [17548.149374] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing set_hostid [17561.818342] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing load_modules_local [17613.102990] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 17152 sec ========= 07:27:27 (1785410847) [17616.663409] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 07:27:30 (1785410850) === [17621.776878] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 07:27:36 (1785410856) === [17681.480117] Lustre: DEBUG MARKER: oleg234-server.virtnet: executing unload_modules_local [17686.479567] Key type lgssc unregistered [17687.017914] LNet: 388994:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [17687.023219] LNetError: 388994:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [17687.064745] LNet: Removed LNI 192.168.202.134@tcp [17688.517486] Key type .llcrypt unregistered [17688.521880] Key type ._llcrypt unregistered