[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 477463128 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.003371] x2apic enabled [ 0.004010] Switched APIC routing to physical x2apic. [ 0.005014] kvm-guest: setup PV IPIs [ 0.007865] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008021] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009024] pid_max: default: 32768 minimum: 301 [ 0.010128] LSM: Security Framework initializing [ 0.011045] Yama: becoming mindful. [ 0.012050] SELinux: Initializing. [ 0.013072] *** VALIDATE selinux *** [ 0.021534] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026181] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027127] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028122] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030016] *** VALIDATE tmpfs *** [ 0.031226] *** VALIDATE proc *** [ 0.032212] *** VALIDATE cgroup *** [ 0.033011] *** VALIDATE cgroup2 *** [ 0.034268] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035175] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036007] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037031] Spectre V2 : User space: Vulnerable [ 0.038010] Speculative Store Bypass: Vulnerable [ 0.040886] debug: unmapping init [mem 0xffffffffa5659000-0xffffffffa5660fff] [ 0.042171] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.043699] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.044023] ... version: 2 [ 0.045014] ... bit width: 48 [ 0.046013] ... generic registers: 4 [ 0.047000] ... value mask: 0000ffffffffffff [ 0.047018] ... max period: 00007fffffffffff [ 0.048017] ... fixed-purpose events: 3 [ 0.049016] ... event mask: 000000070000000f [ 0.050293] rcu: Hierarchical SRCU implementation. [ 0.052488] smp: Bringing up secondary CPUs ... [ 0.053505] x86: Booting SMP configuration: [ 0.054034] .... node #0, CPUs: #1 #2 #3 [ 0.057414] smp: Brought up 1 node, 4 CPUs [ 0.059014] smpboot: Max logical packages: 1 [ 0.060018] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.132617] node 0 deferred pages initialised in 69ms [ 0.136149] devtmpfs: initialized [ 0.137241] x86/mm: Memory block size: 128MB [ 0.139899] gcov: version magic: 0x41383552 [ 0.141293] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.142096] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.143211] pinctrl core: initialized pinctrl subsystem [ 0.144185] [ 0.144671] ************************************************************* [ 0.145012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.146017] ** ** [ 0.147015] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.148015] ** ** [ 0.149015] ** This means that this kernel is built to expose internal ** [ 0.150016] ** IOMMU data structures, which may compromise security on ** [ 0.151015] ** your system. ** [ 0.152013] ** ** [ 0.153012] ** If you see this message and you are not debugging the ** [ 0.154015] ** kernel, report this immediately to your vendor! ** [ 0.155013] ** ** [ 0.156011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.157015] ************************************************************* [ 0.158734] NET: Registered protocol family 16 [ 0.159460] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.160062] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.161060] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.162414] cpuidle: using governor menu [ 0.164967] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.168460] PCI: Using configuration type 1 for base access [ 0.170120] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.179066] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.182104] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.186033] cryptd: max_cpu_qlen set to 1000 [ 0.189257] ACPI: Added _OSI(Module Device) [ 0.191007] ACPI: Added _OSI(Processor Device) [ 0.192017] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.193010] ACPI: Added _OSI(Processor Aggregator Device) [ 0.197996] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.204565] ACPI: Interpreter enabled [ 0.205063] ACPI: PM: (supports S0 S3 S4 S5) [ 0.206013] ACPI: Using IOAPIC for interrupt routing [ 0.207106] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.208537] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.218000] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.221034] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.223018] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.227075] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.231474] acpiphp: Slot [2] registered [ 0.233119] acpiphp: Slot [5] registered [ 0.234134] acpiphp: Slot [6] registered [ 0.236106] acpiphp: Slot [7] registered [ 0.237105] acpiphp: Slot [8] registered [ 0.239085] acpiphp: Slot [9] registered [ 0.240173] acpiphp: Slot [10] registered [ 0.241108] acpiphp: Slot [3] registered [ 0.242101] acpiphp: Slot [4] registered [ 0.244101] acpiphp: Slot [11] registered [ 0.245111] acpiphp: Slot [12] registered [ 0.247102] acpiphp: Slot [13] registered [ 0.248103] acpiphp: Slot [14] registered [ 0.250124] acpiphp: Slot [15] registered [ 0.251125] acpiphp: Slot [16] registered [ 0.253102] acpiphp: Slot [17] registered [ 0.254104] acpiphp: Slot [18] registered [ 0.256111] acpiphp: Slot [19] registered [ 0.257104] acpiphp: Slot [20] registered [ 0.259106] acpiphp: Slot [21] registered [ 0.261134] acpiphp: Slot [22] registered [ 0.262109] acpiphp: Slot [23] registered [ 0.263107] acpiphp: Slot [24] registered [ 0.265100] acpiphp: Slot [25] registered [ 0.267110] acpiphp: Slot [26] registered [ 0.268117] acpiphp: Slot [27] registered [ 0.270174] acpiphp: Slot [28] registered [ 0.272134] acpiphp: Slot [29] registered [ 0.273112] acpiphp: Slot [30] registered [ 0.274096] acpiphp: Slot [31] registered [ 0.275089] PCI host bridge to bus 0000:00 [ 0.277019] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.279023] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.281027] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.284029] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.286019] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.289024] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.290190] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.293085] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.297397] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.307017] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.313013] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.315066] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.318018] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.320018] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.323287] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.326645] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.328037] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.331886] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.338017] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.350016] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.357016] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.362369] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.369015] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.375015] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.392016] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.402163] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.411016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.417015] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.437017] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.447593] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.460017] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.467014] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.491017] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.500878] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.511015] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.520017] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.541016] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.554673] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.563015] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.577016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.609015] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.618928] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.629017] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.635014] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.651016] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.661623] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.665427] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.668384] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.670365] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.673309] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.677220] iommu: Default domain type: Passthrough [ 0.679463] SCSI subsystem initialized [ 0.680083] ACPI: bus type USB registered [ 0.682111] usbcore: registered new interface driver usbfs [ 0.683084] usbcore: registered new interface driver hub [ 0.685056] usbcore: registered new device driver usb [ 0.686141] pps_core: LinuxPPS API ver. 1 registered [ 0.688010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.691063] PTP clock support registered [ 0.693090] EDAC MC: Ver: 3.0.0 [ 0.695049] PCI: Using ACPI for IRQ routing [ 0.697300] NetLabel: Initializing [ 0.698013] NetLabel: domain hash size = 128 [ 0.700009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.701075] NetLabel: unlabeled traffic allowed by default [ 0.704117] vgaarb: loaded [ 0.706291] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.708011] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.714000] clocksource: Switched to clocksource kvm-clock [ 0.821975] VFS: Disk quotas dquot_6.6.0 [ 0.823544] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.826428] *** VALIDATE ramfs *** [ 0.827969] *** VALIDATE hugetlbfs *** [ 0.829748] pnp: PnP ACPI init [ 0.832347] pnp: PnP ACPI: found 6 devices [ 0.852854] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.856506] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.858474] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.860817] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.863238] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.865735] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.869237] NET: Registered protocol family 2 [ 0.871851] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.876734] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.880865] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.886909] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.890833] TCP: Hash tables configured (established 65536 bind 65536) [ 0.893378] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.896153] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.899106] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.902035] NET: Registered protocol family 1 [ 0.905248] RPC: Registered named UNIX socket transport module. [ 0.907507] RPC: Registered udp transport module. [ 0.909226] RPC: Registered tcp transport module. [ 0.910940] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.913496] NET: Registered protocol family 44 [ 0.914938] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.916832] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.919132] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.921678] PCI: CLS 0 bytes, default 64 [ 0.923247] Unpacking initramfs... [ 2.332200] debug: unmapping init [mem 0xffff92c1bcc54000-0xffff92c1bffbffff] [ 2.336327] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.338916] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.341214] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.874990] Initialise system trusted keyrings [ 2.877545] Key type blacklist registered [ 2.879288] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.889414] zbud: loaded [ 2.892670] *** VALIDATE nfs *** [ 2.894176] *** VALIDATE nfs4 *** [ 2.896156] pstore: using deflate compression [ 2.899826] Platform Keyring initialized [ 3.010611] NET: Registered protocol family 38 [ 3.012512] Key type asymmetric registered [ 3.014054] Asymmetric key parser 'x509' registered [ 3.015755] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.018507] io scheduler mq-deadline registered [ 3.020346] io scheduler kyber registered [ 3.022507] io scheduler bfq registered [ 3.025730] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.029492] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.033437] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.037753] ACPI: Power Button [PWRF] [ 3.043186] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.050593] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.077144] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.089176] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.117113] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.147709] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.175749] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.181597] Non-volatile memory driver v1.3 [ 3.183959] Linux agpgart interface v0.103 [ 3.217504] virtio_blk virtio1: [vda] 147160 512-byte logical blocks (75.3 MB/71.9 MiB) [ 3.220350] vda: detected capacity change from 0 to 75345920 [ 3.237622] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.241474] vdb: detected capacity change from 0 to 1073741824 [ 3.262061] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.269122] vdc: detected capacity change from 0 to 2621440000 [ 3.282817] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.285837] vdd: detected capacity change from 0 to 2621440000 [ 3.302824] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.306085] vde: detected capacity change from 0 to 4294967296 [ 3.321127] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.325623] vdf: detected capacity change from 0 to 4294967296 [ 3.335456] libphy: Fixed MDIO Bus: probed [ 3.342142] usbcore: registered new interface driver usbserial_generic [ 3.344777] usbserial: USB Serial support registered for generic [ 3.347910] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.352821] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.354853] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.358246] mousedev: PS/2 mouse device common for all mice [ 3.362286] rtc_cmos 00:05: RTC can wake from S4 [ 3.365747] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.371602] rtc_cmos 00:05: registered as rtc0 [ 3.374146] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.377657] intel_pstate: CPU model not supported [ 3.382136] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.382959] hid: raw HID events driver (C) Jiri Kosina [ 3.387962] usbcore: registered new interface driver usbhid [ 3.390630] usbhid: USB HID core driver [ 3.392916] drop_monitor: Initializing network drop monitor service [ 3.395901] Initializing XFRM netlink socket [ 3.397150] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.398094] NET: Registered protocol family 10 [ 3.403785] Segment Routing with IPv6 [ 3.405484] NET: Registered protocol family 17 [ 3.407982] mpls_gso: MPLS GSO support [ 3.414201] RAS: Correctable Errors collector initialized. [ 3.416418] AVX version of gcm_enc/dec engaged. [ 3.418212] AES CTR mode by8 optimization enabled [ 3.507438] sched_clock: Marking stable (3507413558, 0)->(4472357663, -964944105) [ 3.511225] registered taskstats version 1 [ 3.513453] Loading compiled-in X.509 certificates [ 3.515561] zswap: loaded using pool lzo/zbud [ 3.548618] Key type big_key registered [ 3.564769] Key type encrypted registered [ 3.566565] ima: No TPM chip found, activating TPM-bypass! [ 3.569207] ima: Allocated hash algorithm: sha1 [ 3.571483] ima: No architecture policies found [ 3.573872] evm: Initialising EVM extended attributes: [ 3.576450] evm: security.selinux [ 3.577985] evm: security.ima [ 3.579407] evm: security.capability [ 3.581167] evm: HMAC attrs: 0x1 [ 3.584120] rtc_cmos 00:05: setting system clock to 2026-09-11 12:17:32 UTC (1789129052) [ 3.591216] debug: unmapping init [mem 0xffffffffa6603000-0xffffffffa67fffff] [ 3.594416] debug: unmapping init [mem 0xffffffffa5382000-0xffffffffa5658fff] [ 3.603086] Write protecting the kernel read-only data: 28672k [ 3.606450] debug: unmapping init [mem 0xffffffffa3a03000-0xffffffffa3bfffff] [ 3.609280] debug: unmapping init [mem 0xffffffffa4314000-0xffffffffa43fffff] [ 3.641754] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.649430] systemd[1]: Detected virtualization kvm. [ 3.650810] systemd[1]: Detected architecture x86-64. [ 3.652170] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.681393] systemd[1]: No hostname configured. [ 3.683295] systemd[1]: Set hostname to . [ 3.685235] random: systemd: uninitialized urandom read (16 bytes read) [ 3.687517] systemd[1]: Initializing machine ID from random generator. [ 3.743567] random: ln: uninitialized urandom read (6 bytes read) [ 3.837654] random: systemd: uninitialized urandom read (16 bytes read) [ 3.840814] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 3.845456] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 3.853882] systemd[1]: Reached target Paths. [ OK ] Reached target Paths. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Slices. [ OK ] Listening on udev Control Socket. Starting Setup Virtual Console... [ OK ] Reached target Local File Systems. Starting Journal Service... [ OK ] Reached target Swap. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Local Encrypted Volumes. Starting Create Volatile Files and Directories... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. Starting Apply Kernel Variables... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.484261] device-mapper: uevent: version 1.0.3 [ 4.486452] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... [ 4.827108] random: fast init done Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.152864] virtio_net virtio0 ens2: renamed from eth0 [ 5.235306] scsi host0: ata_piix [ 5.270327] scsi host1: ata_piix [ 5.335697] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.339289] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.377035] dracut-initqueue[597]: RTNETLINK answers: File exists [ 10.086453] random: crng init done [ 10.087309] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.402986] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.503020] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.770891] SELinux: Disabled at runtime. [ 11.838565] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.848474] systemd[1]: Detected virtualization kvm. [ 11.850345] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.378658] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.381718] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.386226] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.389817] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.392889] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.401450] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.417816] systemd[1]: Listening on Process Core Dump Socket. [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-serial\x2dgetty.slice. Mounting POSIX Message Queue File System... [ OK ] Listening on udev Control Socket. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. Mounting Huge Pages File System... Starting Remount Root and Kernel File Systems... Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on initctl Compatibility Named Pipe. Starting udev Coldplug all Devices... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ 12.497382] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Apply Kernel Variables... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Mounting Kernel Debug File System... [ OK ] Stopped target Initrd Root File System. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-getty.slice. [ OK ] Reached target rpc_pipefs.target. [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Kernel Debug File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 12.803253] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.165921] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.178681] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.270043] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.283263] EDAC sbridge: Ver: 1.1.2 [ 14.930770] Key type dns_resolver registered [ 15.260117] NFS: Registering the id_resolver key type [ 15.262254] Key type id_resolver registered [ 15.263682] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started irqbalance daemon. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. Starting Restore /run/initramfs on shutdown... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg643-server login: [ 42.367496] libcfs: loading out-of-tree module taints kernel. [ 42.391772] Key type ._llcrypt registered [ 42.393725] Key type .llcrypt registered [ 42.445159] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_hostid [ 61.460146] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 63.440851] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 63.468665] alg: No test for adler32 (adler32-zlib) [ 65.110593] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 66.109765] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [ 67.855233] Key type lgssc registered [ 69.797808] Lustre: Echo OBD driver; http://www.lustre.org/ [ 89.444266] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 107.252014] hrtimer: interrupt took 11019955 ns [ 131.398685] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 145.014539] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 145.062310] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 146.335786] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 146.361170] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 146.458981] Lustre: lustre-MDT0000: new disk, initializing [ 146.537388] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 146.553655] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 150.739408] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 165.447533] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 165.563520] Lustre: 6519:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 165.599457] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 165.605741] Lustre: Skipped 1 previous similar message [ 165.799958] Lustre: lustre-MDT0001: new disk, initializing [ 165.881821] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 165.913500] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 165.954886] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 170.784210] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 175.645602] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 185.740089] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 186.063429] Lustre: lustre-OST0000: new disk, initializing [ 186.067664] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 186.075622] Lustre: 8455:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 186.181764] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 193.063659] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 193.554902] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 193.572138] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 193.658776] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 209.687034] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 209.898353] Lustre: lustre-OST0001: new disk, initializing [ 209.901589] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 209.909212] Lustre: 9528:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 210.000105] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 216.642592] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 216.654467] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 216.703404] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 217.207213] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 229.987934] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 236.060489] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 243.122591] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing check_logdir /tmp/testlogs/ [ 248.323500] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing yml_node [ 253.818194] Lustre: DEBUG MARKER: Client: 2.17.57.133 [ 256.606981] Lustre: DEBUG MARKER: MDS: 2.17.57.133 [ 259.065590] Lustre: DEBUG MARKER: OSS: 2.17.57.133 [ 260.821241] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri Sep 11 08:21:46 EDT 2026 [ 278.293689] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 279.919339] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 298.473296] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 298.488338] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 298.508553] Lustre: Skipped 3 previous similar messages [ 298.524141] Lustre: Skipped 3 previous similar messages [ 303.589118] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 303.603934] Lustre: Skipped 3 previous similar messages [ 308.709042] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 308.715328] Lustre: Skipped 3 previous similar messages [ 310.239184] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 310.440348] Lustre: server umount lustre-MDT0000 complete [ 314.221903] LustreError: 6512:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789129363 with bad export cookie 6145516635498242076 [ 314.229751] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 314.232697] LustreError: 6512:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 318.947959] LustreError: 6527:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 318.958803] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 318.961082] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 318.973044] LustreError: 6527:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [ 319.008740] Lustre: Skipped 1 previous similar message [ 319.971518] LustreError: 9536:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 319.995488] LustreError: 9536:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 320.800507] Lustre: server umount lustre-MDT0001 complete [ 324.248693] Lustre: server umount lustre-OST0000 complete [ 328.398388] Lustre: server umount lustre-OST0001 complete [ 335.500562] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_hostid [ 345.482988] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 379.057694] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 379.420868] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 379.475749] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 379.549095] Lustre: lustre-MDT0000: new disk, initializing [ 379.625154] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 379.641552] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 383.871547] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 394.337918] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 394.429963] Lustre: 15083:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 394.522242] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 394.527337] Lustre: Skipped 1 previous similar message [ 394.614856] Lustre: lustre-MDT0001: new disk, initializing [ 394.740102] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 394.779504] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 394.793406] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 399.006897] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 407.866848] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 412.928167] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 418.684274] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 418.921925] Lustre: lustre-OST0000: new disk, initializing [ 418.926162] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 418.932587] Lustre: 17051:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 419.016648] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 420.172148] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 420.186175] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 420.301323] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 424.721294] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 433.436967] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 438.045413] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 438.304765] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 443.319850] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 443.568563] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 445.921861] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 445.944849] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 445.976379] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 445.990672] Lustre: Skipped 4 previous similar messages [ 451.781867] Lustre: server umount lustre-OST0000 complete [ 456.160750] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 456.164387] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 456.173666] Lustre: Skipped 2 previous similar messages [ 456.175339] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 456.178300] Lustre: Skipped 2 previous similar messages [ 456.180784] LustreError: Skipped 1 previous similar message [ 461.608914] Lustre: server umount lustre-MDT0000 complete [ 465.630880] LustreError: 15074:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789129514 with bad export cookie 6145516635498243609 [ 465.647565] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 465.648167] LustreError: 15074:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 466.102439] Lustre: server umount lustre-MDT0001 complete [ 471.694492] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 08:25:17 (1789129517) [ 478.653928] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 479.115347] LustreError: 19315:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 479.189490] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 483.754428] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 484.320980] LustreError: 19316:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 489.440195] LustreError: 19315:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 492.293748] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 497.768543] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 505.540077] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 511.620679] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 517.956416] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 523.866199] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 531.650491] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 544.256540] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 553.444383] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 553.456491] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 553.462971] Lustre: Skipped 2 previous similar messages [ 553.472651] Lustre: Skipped 4 previous similar messages [ 556.991980] Lustre: server umount lustre-MDT0000 complete [ 558.564717] LustreError: 19310:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 558.594248] LustreError: 19310:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 560.640105] LustreError: 19295:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789129609 with bad export cookie 6145516635498244491 [ 560.641313] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 560.655212] LustreError: 19295:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 561.079395] Lustre: server umount lustre-MDT0001 complete [ 575.017528] Lustre: server umount lustre-OST0000 complete [ 596.227198] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 609.127225] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 609.493919] LustreError: 23716:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 609.646518] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 609.651257] Lustre: Skipped 2 previous similar messages [ 616.726856] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 626.387221] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 630.741918] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 633.236902] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 641.136238] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 646.014228] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 654.532356] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 654.722153] Lustre: lustre-OST0001: new disk, initializing [ 654.731414] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 654.739268] Lustre: 26031:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 656.567183] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 656.578885] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 656.739602] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 661.431980] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 670.991905] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 675.019493] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 687.599617] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 687.611382] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 687.621096] Lustre: Skipped 2 previous similar messages [ 687.643982] Lustre: Skipped 3 previous similar messages [ 693.963455] Lustre: server umount lustre-MDT0000 complete [ 697.826450] LustreError: 23711:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 697.860132] LustreError: 23711:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [ 697.928510] LustreError: 23696:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789129746 with bad export cookie 6145516635498245338 [ 697.939758] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 697.941934] LustreError: 23696:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 698.261389] Lustre: server umount lustre-MDT0001 complete [ 711.379666] Lustre: server umount lustre-OST0000 complete [ 724.528772] Lustre: server umount lustre-OST0001 complete [ 741.946563] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 750.771633] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 751.241981] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 751.249169] Lustre: Skipped 3 previous similar messages [ 756.057797] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 761.829220] LustreError: 29054:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 761.843909] LustreError: 29054:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 764.608924] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 770.726664] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 773.911284] Lustre: 30201:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 780.010215] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 787.853456] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 798.042401] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 805.575723] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 812.753635] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 816.654211] Lustre: 32080:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 824.289732] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 824.298996] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 824.314182] Lustre: Skipped 4 previous similar messages [ 824.326677] Lustre: Skipped 6 previous similar messages [ 828.860983] Lustre: server umount lustre-MDT0000 complete [ 832.721993] LustreError: 29035:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789129881 with bad export cookie 6145516635498246633 [ 832.729720] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 832.735745] LustreError: 29035:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 833.201371] Lustre: server umount lustre-MDT0001 complete [ 847.948928] Lustre: server umount lustre-OST0000 complete [ 850.656152] Lustre: 3652:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789129883/real 1789129883] req@ffff92c1061d3c00 x1876037851189248/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789129899 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 852.380843] Lustre: server umount lustre-OST0001 complete [ 861.707481] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 08:31:47 (1789129907) [ 894.858904] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 904.149908] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 904.611877] LustreError: 34887:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 904.636760] LustreError: 34887:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 8 previous similar messages [ 909.965776] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 919.870368] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 924.728956] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 928.126664] Lustre: 36034:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 935.561915] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 943.021577] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 951.873220] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 959.089847] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 966.598692] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 970.143982] Lustre: 37913:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 976.355941] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 976.374916] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 976.394637] Lustre: Skipped 1 previous similar message [ 976.407346] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 976.416736] Lustre: Skipped 2 previous similar messages [ 982.256960] Lustre: server umount lustre-MDT0000 complete [ 986.400366] LustreError: 34869:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789130035 with bad export cookie 6145516635498247739 [ 986.419930] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 986.424716] LustreError: 34869:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 986.852963] Lustre: server umount lustre-MDT0001 complete [ 1001.786251] Lustre: server umount lustre-OST0000 complete [ 1004.319548] Lustre: 3654:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789130037/real 1789130037] req@ffff92c234a35e00 x1876037851238272/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789130053 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1006.024079] Lustre: server umount lustre-OST0001 complete [ 1017.287589] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 08:34:23 (1789130063) [ 1041.755476] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 1052.553917] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1053.074363] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1053.084131] Lustre: Skipped 7 previous similar messages [ 1057.650277] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 1065.571480] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1070.315698] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 1073.389674] Lustre: 41228:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1079.670049] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1084.854305] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 1091.950938] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1099.519825] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 1107.818459] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1112.447871] Lustre: 43106:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1113.378508] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1113.383479] Lustre: Skipped 1 previous similar message [ 1117.158918] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1117.163365] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1117.174811] Lustre: Skipped 4 previous similar messages [ 1122.730690] Lustre: server umount lustre-MDT0000 complete [ 1126.506261] LustreError: 40059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789130175 with bad export cookie 6145516635498248845 [ 1126.516966] LustreError: 40059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1126.518221] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1133.042216] Lustre: server umount lustre-MDT0001 complete [ 1143.064349] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1156.912170] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 1161.453024] LustreError: 41593:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1161.468058] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1161.476633] LustreError: 41593:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 31 previous similar messages [ 1166.578032] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1170.213778] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 1171.952504] Lustre: lustre-MDT0001-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1171.955895] Lustre: Skipped 1 previous similar message [ 1177.404607] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1179.974581] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1186.180149] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1188.844389] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1197.551640] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1201.464783] Lustre: server umount lustre-MDT0000 complete [ 1205.025809] LustreError: 40057:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789130253 with bad export cookie 6145516635498249972 [ 1205.034618] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1205.045462] LustreError: 40057:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1205.390142] Lustre: server umount lustre-MDT0001 complete [ 1219.540604] Lustre: server umount lustre-OST0000 complete [ 1233.645157] Lustre: server umount lustre-OST0001 complete [ 1243.695779] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 08:38:09 (1789130289) [ 1262.592240] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 1270.970050] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1276.246676] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 1285.009631] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1289.873217] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 1292.826577] Lustre: 48863:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1299.507896] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1305.668182] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 1313.422661] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1319.900531] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 1327.051189] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1331.791133] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1345.113472] Lustre: server umount lustre-MDT0000 complete [ 1347.041881] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1347.051851] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1347.057956] Lustre: Skipped 9 previous similar messages [ 1348.794961] LustreError: 47694:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789130397 with bad export cookie 6145516635498250763 [ 1348.795572] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1348.800389] LustreError: 47694:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1349.154806] Lustre: server umount lustre-MDT0001 complete [ 1363.691158] Lustre: server umount lustre-OST0000 complete [ 1377.602878] Lustre: server umount lustre-OST0001 complete [ 1388.165868] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 08:40:33 (1789130433) [ 1391.416815] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1393.412025] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 08:40:39 (1789130439) [ 1405.265268] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_hostid [ 1415.670658] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 1442.167701] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1442.459127] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1442.559557] Lustre: lustre-MDT0000: new disk, initializing [ 1442.637957] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1447.291384] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 1456.355241] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1462.748433] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1462.972331] Lustre: lustre-OST0000: new disk, initializing [ 1462.979078] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1462.983863] Lustre: Skipped 1 previous similar message [ 1462.989034] Lustre: 54941:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1464.807679] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1464.820456] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1464.857573] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1469.700494] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 1479.333854] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1481.694368] Lustre: DEBUG MARKER: create test files [ 1513.775343] Lustre: server umount lustre-OST0000 complete [ 1527.442925] Lustre: server umount lustre-MDT0000 complete [ 1539.049873] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [ 1541.909464] Key type lgssc unregistered [ 1542.223859] LNet: 56575:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1542.235568] LNetError: 56575:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1542.260747] LNet: Removed LNI 192.168.206.143@tcp [ 1543.132149] Key type .llcrypt unregistered [ 1543.133924] Key type ._llcrypt unregistered [ 1570.912705] Key type ._llcrypt registered [ 1570.914667] Key type .llcrypt registered [ 1571.021559] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 1572.379918] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1572.404815] alg: No test for adler32 (adler32-zlib) [ 1573.536697] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 1573.809546] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [ 1575.479119] Key type lgssc registered [ 1576.610839] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1585.153422] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1585.204310] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1586.924475] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1591.479205] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 1599.358642] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1606.917440] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1607.321852] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1611.443646] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1611.547594] Lustre: 59311:0:(osd_internal.h:1463:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1611.563333] Lustre: 59311:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1611.574235] Lustre: 59311:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1611.588430] Lustre: 59311:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1611.594365] Lustre: 59311:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1611.608326] Lustre: 59311:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1613.536980] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 1621.489956] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1626.875752] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1632.980631] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1667.278291] Lustre: server umount lustre-OST0000 complete [ 1682.018037] Lustre: server umount lustre-MDT0000 complete [ 1695.278221] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [ 1697.633349] Key type lgssc unregistered [ 1697.946792] LNet: 60945:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1697.955121] LNetError: 60945:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1697.970936] LNet: Removed LNI 192.168.206.143@tcp [ 1698.596381] Key type .llcrypt unregistered [ 1698.599123] Key type ._llcrypt unregistered [ 1728.083921] Key type ._llcrypt registered [ 1728.089482] Key type .llcrypt registered [ 1728.198617] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 1729.370830] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1729.398206] alg: No test for adler32 (adler32-zlib) [ 1730.499393] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 1730.737731] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [ 1732.423673] Key type lgssc registered [ 1733.479614] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1740.929068] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1740.951072] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1742.245316] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1746.519837] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 1754.082366] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1760.695435] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1761.142203] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1767.273345] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1767.393828] Lustre: 63775:0:(osd_internal.h:1463:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 512 < left 544, rollback = 9 [ 1767.403400] Lustre: 63775:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/3, destroy: 0/0/0 [ 1767.410050] Lustre: 63775:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1767.416155] Lustre: 63775:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1767.421906] Lustre: 63775:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1767.428744] Lustre: 63775:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1767.723592] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 1775.720652] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1781.302660] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1793.006851] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1793.026282] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1793.045098] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1796.941718] Lustre: server umount lustre-OST0000 complete [ 1801.765835] Lustre: server umount lustre-MDT0000 complete [ 1814.150866] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [ 1817.119769] Key type lgssc unregistered [ 1817.464589] LNet: 65336:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1817.478184] LNetError: 65336:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1817.493772] LNet: Removed LNI 192.168.206.143@tcp [ 1818.387441] Key type .llcrypt unregistered [ 1818.389716] Key type ._llcrypt unregistered [ 1834.660048] Key type ._llcrypt registered [ 1834.662665] Key type .llcrypt registered [ 1834.756676] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_hostid [ 1850.467048] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 1851.409671] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1851.462459] alg: No test for adler32 (adler32-zlib) [ 1852.685411] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 1853.014874] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [ 1854.807124] Key type lgssc registered [ 1856.303828] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1896.834323] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1896.872237] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1898.063179] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1898.088057] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1898.204600] Lustre: lustre-MDT0000: new disk, initializing [ 1898.326044] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1898.352273] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1902.915788] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 1912.984598] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1913.090339] Lustre: 68488:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1913.127955] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1913.131578] Lustre: Skipped 1 previous similar message [ 1913.186741] Lustre: lustre-MDT0001: new disk, initializing [ 1913.269178] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1913.315259] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1913.327778] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1918.132205] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 1927.962394] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1933.747847] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1941.006341] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1941.336276] Lustre: lustre-OST0000: new disk, initializing [ 1941.343498] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1941.357454] Lustre: 70455:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1941.515981] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1947.245978] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 1949.775239] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1949.785646] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1949.901456] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1957.075422] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1962.399908] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1962.584353] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1966.497474] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1966.697649] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1970.658267] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1970.660578] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1970.677965] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1970.698339] Lustre: Skipped 1 previous similar message [ 1975.298217] Lustre: server umount lustre-OST0000 complete [ 1979.878622] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1979.883770] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1979.893640] Lustre: Skipped 1 previous similar message [ 1979.898996] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1980.902288] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1980.902552] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1980.927546] Lustre: Skipped 1 previous similar message [ 1985.329901] Lustre: server umount lustre-MDT0000 complete [ 1989.033654] LustreError: 69291:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789131037 with bad export cookie 8002029488135424155 [ 1989.034315] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1989.047542] LustreError: 69291:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1989.317948] Lustre: server umount lustre-MDT0001 complete [ 1998.267867] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 08:50:44 (1789131044) [ 2009.960996] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 2052.225575] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_hostid [ 2061.792983] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 2094.666848] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2094.873354] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2094.894321] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2094.975170] Lustre: lustre-MDT0000: new disk, initializing [ 2095.029849] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2095.041186] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2098.920163] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 2108.337434] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2108.488852] Lustre: 76477:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2108.578608] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2108.584460] Lustre: Skipped 1 previous similar message [ 2108.662972] Lustre: lustre-MDT0001: new disk, initializing [ 2108.764741] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2108.807764] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2108.826275] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2113.684046] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 2123.954603] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2130.830822] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2137.534445] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2137.794480] Lustre: lustre-OST0000: new disk, initializing [ 2137.800332] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2137.813506] Lustre: 78446:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2137.921513] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2144.200648] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 2144.326259] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2144.338541] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2144.463895] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2153.972646] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2158.580667] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2158.770085] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2163.166222] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2163.395486] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2165.219458] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2165.223772] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2165.239356] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2165.257962] Lustre: Skipped 1 previous similar message [ 2170.341028] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2170.354787] Lustre: Skipped 1 previous similar message [ 2171.327839] Lustre: server umount lustre-OST0000 complete [ 2175.457129] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2175.459450] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2175.481794] Lustre: Skipped 2 previous similar messages [ 2180.580033] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2180.592674] Lustre: Skipped 3 previous similar messages [ 2181.364815] Lustre: server umount lustre-MDT0000 complete [ 2184.743165] LustreError: 76469:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789131233 with bad export cookie 8002029488135425044 [ 2184.749128] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2184.753669] LustreError: 76469:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2185.031585] Lustre: server umount lustre-MDT0001 complete [ 2194.574653] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 08:54:00 (1789131240) [ 2201.345027] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2201.787864] LustreError: 80855:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2201.800990] LustreError: 80855:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 2201.864468] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2206.134995] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 2207.207141] LustreError: 80856:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2211.314988] LustreError: 80855:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2214.866315] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2219.750902] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 2227.469930] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2233.157232] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2239.636902] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2239.908885] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2239.917630] Lustre: Skipped 1 previous similar message [ 2245.756966] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 2253.726990] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2258.232231] Lustre: 83525:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 5ecec200-c647-413d-9054-bf066a878f31 at adminstrative request [ 2262.502756] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2262.513133] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2272.744952] Lustre: MGS: Client 5ecec200-c647-413d-9054-bf066a878f31 (at 0@lo) reconnecting [ 2272.759528] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x6f0ceb6c42834f7f to 0x6f0ceb6c42835226 [ 2272.794627] Lustre: MGC192.168.206.143@tcp: Connection restored to 0@lo (at 0@lo) [ 2295.062789] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2295.348199] Lustre: lustre-OST0001: new disk, initializing [ 2295.350805] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2295.358790] Lustre: 83922:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2301.702198] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 2304.050399] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2304.058590] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2304.163770] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2311.555144] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2314.728575] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2314.730770] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2314.744189] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2314.753727] Lustre: Skipped 1 previous similar message [ 2314.770780] Lustre: Skipped 1 previous similar message [ 2316.783575] Lustre: server umount lustre-OST0001 complete [ 2319.848429] LustreError: 82678:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2319.875463] LustreError: 82678:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 2322.916875] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2322.923904] LustreError: Skipped 1 previous similar message [ 2324.965244] LustreError: 82679:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2324.986945] LustreError: 82679:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 2326.869135] Lustre: server umount lustre-OST0000 complete [ 2333.152406] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2333.163128] Lustre: Skipped 2 previous similar messages [ 2336.771390] Lustre: server umount lustre-MDT0000 complete [ 2340.320172] LustreError: 83549:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2340.337636] LustreError: 83549:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 2340.922057] LustreError: 80836:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789131389 with bad export cookie 8002029488135426598 [ 2340.933260] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2340.937495] LustreError: 80836:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2341.298755] Lustre: server umount lustre-MDT0001 complete [ 2350.005613] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 08:56:36 (1789131396) [ 2351.493697] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2352.979926] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 08:56:39 (1789131399) [ 2354.258098] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2355.893468] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 08:56:42 (1789131402) [ 2357.465518] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2359.176815] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 08:56:45 (1789131405) [ 2360.696182] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2362.156641] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 08:56:48 (1789131408) [ 2363.705196] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2369.739626] loop: module loaded [ 2369.755053] loop0: detected capacity change from 0 to 4294967296 [ 2369.812455] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2371.317323] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2375.530896] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2379.868239] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2405.024841] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2405.096304] LustreError: 88043:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2405.102429] LustreError: 88043:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2405.115268] LustreError: 88043:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2405.120397] LustreError: 88043:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2423.024218] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_hostid [ 2432.092422] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 2466.910404] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2467.187479] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2467.209868] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2467.292336] Lustre: lustre-MDT0000: new disk, initializing [ 2467.382959] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2467.386966] Lustre: Skipped 1 previous similar message [ 2467.416691] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2472.147311] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 2482.719858] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2482.872398] Lustre: 90905:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2483.152391] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2487.977705] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 2497.956541] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2503.591489] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2509.714516] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2509.952035] Lustre: 92875:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2516.037227] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 2518.568894] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2518.705152] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2524.755456] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2529.135801] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2529.362207] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2533.383199] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2533.600190] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2538.976634] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2538.984505] LustreError: Skipped 2 previous similar messages [ 2538.989738] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2538.998700] Lustre: Skipped 1 previous similar message [ 2539.004613] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2539.010276] Lustre: Skipped 7 previous similar messages [ 2541.562339] Lustre: server umount lustre-OST0000 complete [ 2551.397474] Lustre: server umount lustre-MDT0000 complete [ 2554.853047] LustreError: 92093:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2554.879430] LustreError: 92093:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 2555.094179] LustreError: 92885:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789131603 with bad export cookie 8002029488135427214 [ 2555.097209] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2555.104827] LustreError: 92885:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2555.413838] Lustre: server umount lustre-MDT0001 complete [ 2565.218439] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 09:00:10 (1789131610) [ 2571.983689] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2577.409380] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 2585.685576] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2590.340980] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 2597.226298] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2603.105839] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2609.948984] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2610.359240] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2610.364413] Lustre: Skipped 4 previous similar messages [ 2616.659929] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 2624.440358] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2629.208609] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2629.441854] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2633.095064] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2633.341404] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2639.411991] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2639.605649] Lustre: lustre-OST0001: new disk, initializing [ 2639.616742] Lustre: Skipped 2 previous similar messages [ 2639.624746] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2639.630937] Lustre: Skipped 3 previous similar messages [ 2639.641243] Lustre: 98676:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2645.305473] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 2646.641733] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2646.649251] Lustre: Skipped 2 previous similar messages [ 2646.657720] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2646.761668] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2654.213747] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2658.374412] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2658.541260] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2662.780790] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2662.940585] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2683.028818] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2709.447546] Lustre: Failing over lustre-MDT0000 [ 2709.848260] LustreError: 100776:0:(ldlm_resource.c:1207:ldlm_resource_complain()) MGS: namespace resource [0x65727473756c:0x0:0x0].0x0 (000000009e6b039b) refcount nonzero (1) after lock cleanup; forcing cleanup. [ 2712.007278] Lustre: server umount lustre-MDT0000 complete [ 2712.204856] LustreError: 95281:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.206.43@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2712.236094] LustreError: 95281:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 2713.579889] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2713.589335] Lustre: Skipped 6 previous similar messages [ 2730.463146] Lustre: 66714:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789131762/real 1789131762] req@0000000013edc8c6 x1876039723918464/t0(0) o400->MGC192.168.206.143@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1789131778 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2730.485469] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2737.223697] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2737.227733] LDISKFS-fs (dm-0): recovery complete [ 2737.248484] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2740.711511] LustreError: 66711:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000d7641a3d x1876039723927936/t0(0) o250->MGC192.168.206.143@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2741.164298] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2742.884327] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2742.888836] Lustre: 96723:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2745.758828] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 2746.346959] Lustre: 95285:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2746.359614] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 2746.362782] Lustre: 95285:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 1 previous similar message [ 2746.452337] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2746.775170] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2747.105541] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2747.126575] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 1 previous similar message [ 2747.455105] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2747.459438] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 1 previous similar message [ 2747.466529] Lustre: 101528:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2747.476271] Lustre: 101528:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 7 previous similar messages [ 2748.122178] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2748.135117] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 2 previous similar messages [ 2748.775274] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2748.785299] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 3 previous similar messages [ 2749.772793] Lustre: 101528:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2749.784232] Lustre: 101528:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 6 previous similar messages [ 2750.127934] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2750.136622] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 5 previous similar messages [ 2750.783162] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2750.791308] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 5 previous similar messages [ 2753.964627] Lustre: 101528:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2753.976810] Lustre: 101528:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 12 previous similar messages [ 2754.298389] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2754.322158] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 12 previous similar messages [ 2754.951077] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2754.964723] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 12 previous similar messages [ 2761.992814] Lustre: 101528:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2762.026051] Lustre: 101528:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 24 previous similar messages [ 2762.362890] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2762.390466] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 24 previous similar messages [ 2763.031099] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2763.037217] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 24 previous similar messages [ 2778.304833] Lustre: 101528:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2778.327052] Lustre: 101528:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 50 previous similar messages [ 2778.656841] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2778.675349] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 50 previous similar messages [ 2779.304984] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2779.320798] LustreError: 101528:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 50 previous similar messages [ 2802.151141] Lustre: 101528:0:(ldlm_lib.c:2457:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2802.166873] Lustre: 101528:0:(ldlm_lib.c:2467:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2802.184925] Lustre: 101528:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 8990c9b7-b6c2-46d6-ad12-b2db47365c8f@192.168.206.43@tcp [ 2802.205445] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2802.224627] Lustre: 101528:0:(ldlm_lib.c:1977:abort_req_replay_queue()) @@@ aborted: req@00000000e8762bdd x1876039706939392/t0(8589934771) o101->8990c9b7-b6c2-46d6-ad12-b2db47365c8f@192.168.206.43@tcp:56/0 lens 592/0 e 0 to 0 dl 1789131861 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2802.246378] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2802.274546] Lustre: lustre-MDT0000: Denying connection for new client 8990c9b7-b6c2-46d6-ad12-b2db47365c8f (at 192.168.206.43@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2802.277493] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2802.356463] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2802.362732] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2802.373829] Lustre: Skipped 2 previous similar messages [ 2802.435883] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 2802.437370] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 2821.600828] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2821.606604] LustreError: Skipped 2 previous similar messages [ 2821.609913] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2821.613750] Lustre: Skipped 4 previous similar messages [ 2833.375116] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2833.482841] Lustre: server umount lustre-OST0000 complete [ 2838.508870] LustreError: 101912:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2838.527581] LustreError: 101912:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 39 previous similar messages [ 2843.576832] Lustre: server umount lustre-OST0001 complete [ 2848.736242] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2848.752239] Lustre: Skipped 5 previous similar messages [ 2853.468482] Lustre: server umount lustre-MDT0000 complete [ 2856.865342] LustreError: 95265:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789131905 with bad export cookie 8002029488135478433 [ 2856.874664] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2856.878794] LustreError: 95265:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2857.161321] Lustre: server umount lustre-MDT0001 complete [ 2864.762283] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 09:05:11 (1789131911) [ 2870.604432] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2870.611517] Lustre: *** cfs_fail_loc=197, val=0*** [ 2886.114234] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2891.231455] LustreError: 103503:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.143@tcp: failed processing log, type 4: rc = -110 [ 2896.377526] Lustre: *** cfs_fail_loc=197, val=0*** [ 2896.380866] Lustre: Skipped 65 previous similar messages [ 2916.960275] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2916.970188] Lustre: Skipped 2 previous similar messages [ 2923.726847] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 2932.647993] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2935.106437] Lustre: server umount lustre-OST0000 complete [ 2943.436135] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 09:06:29 (1789131989) [ 2963.787741] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2969.223764] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 2969.574919] LustreError: 105122:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2969.595887] LustreError: 105122:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 2979.564095] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2983.427239] LustreError: 105152:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2983.435659] LustreError: 105152:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2983.441765] LustreError: 105152:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 2983.733929] Lustre: server umount lustre-MDT0000 complete [ 3016.398722] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_hostid [ 3027.195048] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 3062.239386] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3062.433470] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3062.467591] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3062.571957] Lustre: lustre-MDT0000: new disk, initializing [ 3062.657981] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3066.217667] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 3076.120451] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3076.222249] Lustre: 109328:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3076.463358] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3080.497257] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 3089.634320] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3096.198868] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3103.403802] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3103.660720] Lustre: 111295:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3110.136295] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3110.464646] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 3121.860779] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3127.018454] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3127.234721] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3131.790929] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3132.022993] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3135.981694] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3135.983895] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3135.996125] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3135.996144] Lustre: Skipped 10 previous similar messages [ 3135.997969] Lustre: Skipped 1 previous similar message [ 3136.030689] LustreError: Skipped 2 previous similar messages [ 3148.255213] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3148.477976] Lustre: server umount lustre-OST0000 complete [ 3158.671076] Lustre: server umount lustre-MDT0000 complete [ 3161.845343] LustreError: 109320:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789132210 with bad export cookie 8002029488135489024 [ 3161.850880] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3161.857318] LustreError: 109320:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3162.243386] Lustre: server umount lustre-MDT0001 complete [ 3170.074471] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 09:10:16 (1789132216) [ 3178.787458] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [ 3181.768415] Key type lgssc unregistered [ 3182.088967] LNet: 113788:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3182.097962] LNetError: 113788:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3183.145400] LNet: Removed LNI 192.168.206.143@tcp [ 3184.130746] Key type .llcrypt unregistered [ 3184.132802] Key type ._llcrypt unregistered [ 3201.587072] Key type ._llcrypt registered [ 3201.592154] Key type .llcrypt registered [ 3201.687824] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_hostid [ 3216.501260] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 3217.602596] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3217.783443] alg: No test for adler32 (adler32-zlib) [ 3218.879903] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 3219.182946] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [ 3220.911925] Key type lgssc registered [ 3222.252181] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3264.440265] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3264.483383] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3265.760064] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3265.789476] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3265.918606] Lustre: lustre-MDT0000: new disk, initializing [ 3266.018757] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3266.036518] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3270.439885] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 3281.004726] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3281.295988] Lustre: lustre-OST0000: new disk, initializing [ 3281.299560] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3281.307052] Lustre: Skipped 1 previous similar message [ 3281.311941] Lustre: 117963:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3281.408835] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3288.155824] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 3288.179325] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3288.192888] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3288.255478] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3299.261465] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3299.412841] Lustre: lustre-OST0001: new disk, initializing [ 3299.417228] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3299.423858] Lustre: 118827:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3299.502612] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3306.361861] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 3306.670871] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3306.685934] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3306.736228] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3317.219601] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3317.232442] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3317.233179] Lustre: Skipped 1 previous similar message [ 3317.254523] Lustre: Skipped 1 previous similar message [ 3321.258383] Lustre: server umount lustre-MDT0000 complete [ 3327.962558] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3332.603087] LustreError: 118826:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789132381 with bad export cookie 5962952152662012364 [ 3332.622745] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3332.720780] Lustre: server umount lustre-OST0000 complete [ 3336.843209] Lustre: server umount lustre-OST0001 complete [ 3346.571650] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 09:13:12 (1789132392) [ 3369.076739] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 09:13:34 (1789132414) [ 3379.608772] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 3402.087036] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 09:14:08 (1789132448) [ 3441.588323] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_hostid [ 3452.215482] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 3491.997142] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3492.307949] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3492.379417] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3492.526635] Lustre: lustre-MDT0000: new disk, initializing [ 3492.635837] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3492.667246] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3497.146354] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 3506.768483] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3506.910242] Lustre: 126580:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3506.954983] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3506.958078] Lustre: Skipped 1 previous similar message [ 3507.054628] Lustre: lustre-MDT0001: new disk, initializing [ 3507.144857] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3507.170039] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3507.181053] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3511.342076] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 3520.232394] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3525.664709] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3532.079481] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3532.289405] Lustre: lustre-OST0000: new disk, initializing [ 3532.292790] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3532.297293] Lustre: 128546:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3532.386562] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3537.406037] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 3541.053807] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3541.071282] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3541.244147] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3547.407761] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3551.978728] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3552.210756] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3557.187239] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3557.382517] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3561.958048] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3561.964140] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3561.984551] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3565.651587] Lustre: server umount lustre-OST0000 complete [ 3572.194110] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3572.210807] Lustre: Skipped 1 previous similar message [ 3572.217693] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3572.222580] Lustre: Skipped 1 previous similar message [ 3575.407642] Lustre: server umount lustre-MDT0000 complete [ 3578.939719] LustreError: 128556:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789132627 with bad export cookie 5962952152662013589 [ 3578.943656] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3578.949533] LustreError: 128556:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3579.335099] Lustre: server umount lustre-MDT0001 complete [ 3588.826382] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 09:17:14 (1789132634) [ 3595.539756] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3595.980335] LustreError: 130958:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3595.999654] LustreError: 130958:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 3596.067467] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3601.018237] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 3601.380970] LustreError: 130959:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3601.397105] LustreError: 130959:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 3605.476398] LustreError: 130958:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3609.295758] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3614.823352] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 3622.599804] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3629.713966] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3636.280467] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3636.587279] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3636.593668] Lustre: Skipped 1 previous similar message [ 3642.877355] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 3651.067510] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3658.838298] Lustre: *** cfs_fail_loc=159, val=0*** [ 3675.292735] Lustre: lustre-MDT0000: Client 208ab6a9-8e86-4284-be78-36a224ddae24 (at 192.168.206.43@tcp) reconnecting [ 3677.061090] Lustre: *** cfs_fail_loc=159, val=0*** [ 3677.068141] Lustre: Skipped 5 previous similar messages [ 3692.692294] Lustre: lustre-MDT0000: Client 208ab6a9-8e86-4284-be78-36a224ddae24 (at 192.168.206.43@tcp) reconnecting [ 3700.362667] Lustre: server umount lustre-OST0000 complete [ 3706.849435] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3706.858470] LustreError: Skipped 1 previous similar message [ 3706.861092] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3706.876575] Lustre: Skipped 1 previous similar message [ 3706.881239] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3712.483515] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3712.490337] Lustre: Skipped 3 previous similar messages [ 3713.596993] Lustre: server umount lustre-MDT0000 complete [ 3717.460212] LustreError: 130940:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789132766 with bad export cookie 5962952152662014471 [ 3717.464541] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3717.468557] LustreError: 130940:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3727.999633] Lustre: server umount lustre-MDT0001 complete [ 3739.758669] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [ 3742.718882] Key type lgssc unregistered [ 3743.088656] LNet: 134740:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3743.103597] LNetError: 134740:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3743.129280] LNet: Removed LNI 192.168.206.143@tcp [ 3744.269707] Key type .llcrypt unregistered [ 3744.272783] Key type ._llcrypt unregistered [ 3765.978796] Key type ._llcrypt registered [ 3765.981370] Key type .llcrypt registered [ 3766.063885] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 3766.937520] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3767.120555] alg: No test for adler32 (adler32-zlib) [ 3768.281292] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 3768.551557] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [ 3770.303206] Key type lgssc registered [ 3771.366614] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3779.071063] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3779.107085] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3780.497989] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3785.022576] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 3790.820757] LustreError: 136142:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3790.838764] LustreError: 136142:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 3793.607061] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3793.952144] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3798.224757] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 3804.752688] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3810.381766] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3817.382789] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3817.794562] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3824.649865] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 3826.005373] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3832.405728] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3843.171349] Lustre: *** cfs_fail_loc=159, val=0*** [ 3858.575632] Lustre: lustre-MDT0000: Client 7de3c41a-062f-48cd-897d-951131e2a901 (at 192.168.206.43@tcp) reconnecting [ 3860.995187] Lustre: *** cfs_fail_loc=159, val=0*** [ 3876.502544] Lustre: lustre-MDT0001: Client 7de3c41a-062f-48cd-897d-951131e2a901 (at 192.168.206.43@tcp) reconnecting [ 3878.293783] Lustre: *** cfs_fail_loc=159, val=0*** [ 3878.298898] Lustre: Skipped 3 previous similar messages [ 3893.935752] Lustre: lustre-MDT0001: Client 7de3c41a-062f-48cd-897d-951131e2a901 (at 192.168.206.43@tcp) reconnecting [ 3899.310433] Lustre: 139177:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3902.659363] Lustre: *** cfs_fail_loc=159, val=0*** [ 3902.663334] Lustre: Skipped 4 previous similar messages [ 3919.017170] Lustre: lustre-MDT0000: Client 194ddee5-e90f-4542-8814-c976359ff3cb (at 192.168.206.43@tcp) reconnecting [ 3920.680331] Lustre: *** cfs_fail_loc=159, val=0*** [ 3920.684158] Lustre: Skipped 14 previous similar messages [ 3936.934476] Lustre: lustre-MDT0000: Client 194ddee5-e90f-4542-8814-c976359ff3cb (at 192.168.206.43@tcp) reconnecting [ 3938.568737] Lustre: 139468:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3957.216434] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3957.219723] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3957.245683] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3959.269813] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3959.272020] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3959.277531] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3960.693792] Lustre: server umount lustre-OST0000 complete [ 3967.969043] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3967.977142] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3968.000159] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3968.009819] Lustre: Skipped 1 previous similar message [ 3971.054794] Lustre: server umount lustre-MDT0000 complete [ 3974.624502] LustreError: 139418:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3974.639750] LustreError: 139418:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 3975.408772] LustreError: 138957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789133024 with bad export cookie 6110934460385364894 [ 3975.408772] LustreError: 136125:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789133024 with bad export cookie 6110934460385364894 [ 3975.412274] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3975.702795] Lustre: server umount lustre-MDT0001 complete [ 3986.999752] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [ 3989.863284] Key type lgssc unregistered [ 3990.166517] LNet: 140549:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3990.183064] LNetError: 140549:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3990.212305] LNet: Removed LNI 192.168.206.143@tcp [ 3990.908327] Key type .llcrypt unregistered [ 3990.910328] Key type ._llcrypt unregistered [ 4017.107975] Key type ._llcrypt registered [ 4017.112313] Key type .llcrypt registered [ 4017.260296] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 4018.193246] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4018.222291] alg: No test for adler32 (adler32-zlib) [ 4019.233213] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 4019.477385] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [ 4021.175167] Key type lgssc registered [ 4022.407890] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4030.742338] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4030.758371] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4032.208218] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4037.561952] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 4042.725163] LustreError: 141964:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4042.739894] LustreError: 141964:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4046.072259] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4046.493546] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4051.395182] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 4059.326267] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4064.739883] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4071.187447] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4071.331358] LustreError: 143786:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4071.605217] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4077.052891] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4077.409055] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 4079.096551] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4085.344635] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4091.117724] Lustre: 144546:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4094.967948] Lustre: 144595:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4096.672312] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4097.545323] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4101.432639] Lustre: 144741:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4102.419959] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4103.302986] Lustre: 144839:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4109.798019] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4109.800098] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4109.803236] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4109.829650] Lustre: Skipped 1 previous similar message [ 4114.205216] Lustre: server umount lustre-OST0000 complete [ 4120.035628] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4120.039552] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4120.060221] Lustre: Skipped 1 previous similar message [ 4120.088459] Lustre: Skipped 2 previous similar messages [ 4125.155040] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4125.162690] Lustre: Skipped 1 previous similar message [ 4129.248481] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4129.256533] Lustre: Skipped 1 previous similar message [ 4132.319195] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4132.561722] Lustre: server umount lustre-MDT0000 complete [ 4134.371980] LustreError: 142970:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4134.400604] LustreError: 142970:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4136.530847] LustreError: 142699:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789133185 with bad export cookie 9245061663258981104 [ 4136.534048] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4136.546679] LustreError: 142699:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4136.874030] Lustre: server umount lustre-MDT0001 complete [ 4147.785427] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [ 4150.189227] Key type lgssc unregistered [ 4150.498143] LNet: 145905:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4150.508973] LNetError: 145905:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4150.525928] LNet: Removed LNI 192.168.206.143@tcp [ 4151.547183] Key type .llcrypt unregistered [ 4151.549563] Key type ._llcrypt unregistered [ 4176.529130] Key type ._llcrypt registered [ 4176.531806] Key type .llcrypt registered [ 4176.617240] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 4177.409801] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4177.418947] alg: No test for adler32 (adler32-zlib) [ 4178.459715] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 4178.647508] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [ 4180.287160] Key type lgssc registered [ 4181.318928] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4187.980112] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4188.020444] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4189.463852] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4193.386606] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 4199.906919] LustreError: 147321:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4199.937654] LustreError: 147321:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4201.398504] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4201.749829] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4205.964305] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 4213.779320] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4218.585636] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4224.370615] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4224.624251] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4226.667742] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4229.215373] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 4234.869813] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4235.871569] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4242.051586] Lustre: *** cfs_fail_loc=159, val=0*** [ 4258.474298] Lustre: lustre-MDT0000: Client c66b0e6d-487a-45b2-aaf6-f90b94bc44a0 (at 192.168.206.43@tcp) reconnecting [ 4263.919209] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4263.920202] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4263.930600] LustreError: Skipped 2 previous similar messages [ 4263.941799] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4265.216979] Lustre: server umount lustre-OST0000 complete [ 4268.518623] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4268.538334] LustreError: Skipped 2 previous similar messages [ 4268.542487] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4268.558624] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4271.076651] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4271.095389] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4274.905957] Lustre: server umount lustre-MDT0000 complete [ 4276.196788] LustreError: 147315:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4276.227456] LustreError: 147315:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4278.427184] LustreError: 147303:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789133327 with bad export cookie 8255577243133724717 [ 4278.429563] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4278.439053] LustreError: 147303:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4289.183324] Lustre: server umount lustre-MDT0001 complete [ 4300.085387] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [ 4302.989170] Key type lgssc unregistered [ 4303.247521] LNet: 150986:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4303.262441] LNetError: 150986:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4303.285179] LNet: Removed LNI 192.168.206.143@tcp [ 4304.099353] Key type .llcrypt unregistered [ 4304.102718] Key type ._llcrypt unregistered [ 4328.750204] Key type ._llcrypt registered [ 4328.752796] Key type .llcrypt registered [ 4328.837275] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 4329.822973] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4329.991121] alg: No test for adler32 (adler32-zlib) [ 4331.069311] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 4331.414402] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [ 4333.103972] Key type lgssc registered [ 4334.178543] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4342.277559] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4342.297180] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4343.717237] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4348.513371] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 4354.018465] LustreError: 152418:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4354.058234] LustreError: 152418:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4357.927361] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4358.205818] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4363.537747] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 4372.147573] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4378.325934] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4385.247547] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4385.687124] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4388.794732] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4390.552960] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 4397.229747] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4403.140130] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4423.145698] Lustre: lustre-OST0000: evicting ee816b97-c4ba-44d7-816e-dce4ad8c8da1 (at 192.168.206.43@tcp) by administrative request [ 4423.165605] Lustre: Skipped 1 previous similar message [ 4454.368544] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4454.384699] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4454.403081] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4454.406417] Lustre: Skipped 1 previous similar message [ 4457.872021] Lustre: server umount lustre-OST0000 complete [ 4464.610588] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4464.623274] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4464.637333] Lustre: Skipped 1 previous similar message [ 4464.651162] Lustre: Skipped 2 previous similar messages [ 4467.725464] Lustre: server umount lustre-MDT0000 complete [ 4469.732920] LustreError: 152419:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4469.759301] LustreError: 152419:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4471.549451] LustreError: 152398:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789133520 with bad export cookie 2829044438752164181 [ 4471.560966] LustreError: 152398:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4471.563791] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4472.225041] Lustre: server umount lustre-MDT0001 complete [ 4484.205291] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [ 4487.223293] Key type lgssc unregistered [ 4487.662063] LNet: 156493:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4487.691582] LNetError: 156493:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4488.755795] LNet: Removed LNI 192.168.206.143@tcp [ 4489.776511] Key type .llcrypt unregistered [ 4489.781505] Key type ._llcrypt unregistered [ 4579.902950] Key type ._llcrypt registered [ 4579.906635] Key type .llcrypt registered [ 4580.026480] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 4581.253018] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4581.265162] alg: No test for adler32 (adler32-zlib) [ 4582.273851] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 4582.506251] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [ 4584.167176] Key type lgssc registered [ 4585.063220] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4592.626408] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4592.641993] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4594.105702] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4598.763886] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 4603.367674] LustreError: 158987:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4603.376131] LustreError: 158987:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4607.649766] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4608.140203] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4612.452584] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 4622.426675] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4630.092828] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4638.233749] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4638.669218] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4645.471164] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 4653.622709] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4654.076197] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4663.289054] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4673.512617] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4673.518197] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4673.543032] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4673.558826] Lustre: Skipped 1 previous similar message [ 4678.628724] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4678.634573] Lustre: Skipped 1 previous similar message [ 4679.281626] Lustre: server umount lustre-OST0000 complete [ 4683.749536] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4683.773303] Lustre: Skipped 1 previous similar message [ 4683.778379] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4683.786903] Lustre: Skipped 1 previous similar message [ 4688.876667] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4688.890517] LustreError: 158981:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4688.975371] Lustre: server umount lustre-MDT0000 complete [ 4692.558962] LustreError: 158966:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789133741 with bad export cookie 602978943136939147 [ 4692.562225] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4692.572644] LustreError: 158966:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4692.919299] Lustre: server umount lustre-MDT0001 complete [ 4704.505624] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [ 4707.285257] Key type lgssc unregistered [ 4707.598701] LNet: 162557:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4707.610331] LNetError: 162557:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4708.649879] LNet: Removed LNI 192.168.206.143@tcp [ 4709.573458] Key type .llcrypt unregistered [ 4709.582497] Key type ._llcrypt unregistered [ 4736.408198] Key type ._llcrypt registered [ 4736.412684] Key type .llcrypt registered [ 4736.536314] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 4737.239846] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4737.275484] alg: No test for adler32 (adler32-zlib) [ 4738.306734] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 4738.497347] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [ 4740.225167] Key type lgssc registered [ 4741.374762] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4760.019085] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 09:36:46 (1789133806) [ 4777.640360] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_hostid [ 4794.401504] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 4833.422608] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4833.476737] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4834.749695] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4834.779039] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4834.828320] Lustre: lustre-MDT0000: new disk, initializing [ 4834.911297] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4834.938551] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4839.612555] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 4850.828791] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4850.977382] Lustre: 166846:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4851.047751] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4851.053290] Lustre: Skipped 1 previous similar message [ 4851.113421] Lustre: lustre-MDT0001: new disk, initializing [ 4851.192578] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4851.220949] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4851.234255] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4854.794945] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 4864.323921] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4869.776385] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4876.890220] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4877.205915] Lustre: lustre-OST0000: new disk, initializing [ 4877.211442] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4877.224266] Lustre: 168814:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4877.347391] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4883.028253] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4883.037554] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4883.147785] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4884.480434] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 4893.772798] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4903.906642] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4903.913787] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4903.914023] Lustre: Skipped 1 previous similar message [ 4903.916170] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4909.034650] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4909.037399] Lustre: Skipped 2 previous similar messages [ 4909.334324] Lustre: server umount lustre-OST0000 complete [ 4914.147447] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4914.168849] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4914.180847] Lustre: Skipped 1 previous similar message [ 4919.330981] Lustre: server umount lustre-MDT0000 complete [ 4924.167597] LustreError: 166839:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789133973 with bad export cookie 17296778472380342484 [ 4924.170886] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4924.185317] LustreError: 166839:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4924.498931] Lustre: server umount lustre-MDT0001 complete [ 4935.782301] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [ 4937.998888] Key type lgssc unregistered [ 4938.427643] LNet: 170722:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4938.440057] LNetError: 170722:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4938.462781] LNet: Removed LNI 192.168.206.143@tcp [ 4939.341958] Key type .llcrypt unregistered [ 4939.346614] Key type ._llcrypt unregistered [ 4960.757333] Key type ._llcrypt registered [ 4960.762083] Key type .llcrypt registered [ 4960.857501] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 4961.471818] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4961.515318] alg: No test for adler32 (adler32-zlib) [ 4962.549337] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 4962.792383] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [ 4964.535396] Key type lgssc registered [ 4965.385463] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4972.314299] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4972.332621] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4973.801196] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4977.377548] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 4983.267509] LustreError: 172141:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4983.292460] LustreError: 172141:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4984.223319] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4984.515949] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4989.022539] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 4995.915452] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5000.963544] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5007.415104] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5007.756641] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5013.362578] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 5019.059778] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5020.565739] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5027.314448] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5027.338917] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5027.351847] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5027.363436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.380742] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5028.095814] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5028.115193] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5028.123759] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5028.129392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.143160] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5028.970042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.958850] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5029.966455] Lustre: Skipped 1 previous similar message [ 5029.993780] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5030.004620] Lustre: Skipped 1 previous similar message [ 5030.009175] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5030.016520] Lustre: Skipped 1 previous similar message [ 5030.021634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.043262] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5030.047642] Lustre: Skipped 1 previous similar message [ 5030.951070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.014648] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5032.029297] Lustre: Skipped 1 previous similar message [ 5032.055108] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5032.075057] Lustre: Skipped 1 previous similar message [ 5032.093397] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5032.107258] Lustre: Skipped 1 previous similar message [ 5032.115544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.123294] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5032.127363] Lustre: Skipped 1 previous similar message [ 5033.091778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.379106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.464220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.645951] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5036.651788] Lustre: Skipped 3 previous similar messages [ 5036.665789] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5036.678650] Lustre: Skipped 3 previous similar messages [ 5036.684715] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5036.692429] Lustre: Skipped 3 previous similar messages [ 5036.696806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.712209] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5036.721647] Lustre: Skipped 3 previous similar messages [ 5037.657839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.983057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.942728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.864441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.699635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.731552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.841881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.978541] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5044.989022] Lustre: Skipped 7 previous similar messages [ 5045.015606] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5045.030850] Lustre: Skipped 7 previous similar messages [ 5045.041469] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5045.050744] Lustre: Skipped 7 previous similar messages [ 5045.058152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.073106] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5045.085650] Lustre: Skipped 7 previous similar messages [ 5045.897983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.696230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.605649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.577938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.654198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.631655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.596348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.656253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.510542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.434769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.319749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.101938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.901334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.902599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.895659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.891660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.878620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.840703] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5061.855985] Lustre: Skipped 17 previous similar messages [ 5061.877160] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5061.895992] Lustre: Skipped 17 previous similar messages [ 5061.900202] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5061.906903] Lustre: Skipped 17 previous similar messages [ 5061.915869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.930536] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5061.942822] Lustre: Skipped 17 previous similar messages [ 5062.939904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.863737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.820519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.703176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.532486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.457829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.464684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.492778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.371420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.137896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.964572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.902579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.648094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.429683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.421823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.493693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.275325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.178721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.804665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.442860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.105922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.031758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.686442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.336489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.019176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.610593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.459284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.296372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.413265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.154311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.973196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.847729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.060418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.200326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.065735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.857639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.597969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.344266] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5094.350182] Lustre: Skipped 37 previous similar messages [ 5094.365347] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5094.379648] Lustre: Skipped 37 previous similar messages [ 5094.386480] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5094.389582] Lustre: Skipped 37 previous similar messages [ 5094.392137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.400238] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5094.408053] Lustre: Skipped 37 previous similar messages [ 5095.303327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.229289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.986955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.932763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.176155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.151210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.135577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.987356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.933432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.158745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.368772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.378525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.310244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.232461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.150803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.762608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.541473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.347106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.956640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.641503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.474667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.132250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.924733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.688466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.384569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.154929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.186343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.010498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.727496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.523574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.612547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.676874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.438872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.227846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.100565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.916748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.654441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.417574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.090276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.036765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.115191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.274387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.284112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.371385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.281370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.415629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.305854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.117351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.947667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.971141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.808536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.719200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.761974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.853853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.312466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.755340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.114749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.008679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.233390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.501253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.602839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.047684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.200032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.475701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.746283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.524963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.573178] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5158.579940] Lustre: Skipped 66 previous similar messages [ 5158.602820] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5158.622335] Lustre: Skipped 66 previous similar messages [ 5158.627977] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5158.632449] Lustre: Skipped 66 previous similar messages [ 5158.647118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.670199] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5158.687270] Lustre: Skipped 66 previous similar messages [ 5159.895311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.931478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.654729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.501404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.303401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.194503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.102500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.980155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.800897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.953186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.766801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.552849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.272579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.930888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.571365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.285904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.942599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.599537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.306884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.008162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.676630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.569487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.368700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.294947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.188710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.377123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.231260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.466633] LustreError: 172160:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5182.475510] LustreError: 172160:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5182.487772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.473252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.366143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.114257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.757186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.736818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.547961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.330785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.954859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.898841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.589247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.453928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.289943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.504728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.643637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.820702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.774746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.517182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.104952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.318720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.125355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.803754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.570887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.305079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.371565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.487860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.730883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.847868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.924881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.038187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.961490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.149239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.179495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.841574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.723932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.385058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.237536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.242271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.147284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.973651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.088603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.452411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.704483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.074572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.062749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.716087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.508579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.412483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.987498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.591567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.349918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.220900] LustreError: 172160:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5229.229124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.230151] LustreError: 172160:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5230.497735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.182964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.074156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.837770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.557713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.402401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.215918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.106480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.915115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.557928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.553292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.255692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.066498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.471745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.659407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.818821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.604126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.284910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.085543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.758789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.513916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.110633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.685601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.350509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.970759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.782480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.502624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.315610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.119362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.032484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.038436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.967637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.976203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.950168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.870240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.679195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.327355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.007614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.772340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.848094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.930246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.017691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.963973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.927486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.872554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.769150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.978337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.127082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.873161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.703426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.474298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.219589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.102905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.800897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.759200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.792044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.697412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.639103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.707368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.567894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.643962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.548170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.612215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.684788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.903468] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5286.922504] Lustre: Skipped 143 previous similar messages [ 5286.966982] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5286.994935] Lustre: Skipped 143 previous similar messages [ 5287.016972] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5287.032788] Lustre: Skipped 143 previous similar messages [ 5287.058220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.075834] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5287.083358] Lustre: Skipped 143 previous similar messages [ 5288.310678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.397811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.663491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.956321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.014193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.847674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.159699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.098507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.028738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.981516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.883192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.634116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.528609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.451358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.723197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.748044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.655614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.525896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.376170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.208446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.101859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.982509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.885562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.770281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.756515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.630259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.502242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.220288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.006737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.950302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.900940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.696169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.505289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.385952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.243637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.180660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.111441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.060892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.880441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.868209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.679415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.645893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.475541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.336602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.186072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.232727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.172953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.345843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.275855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.268388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.102416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.665234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.429393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.103200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.805454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.511993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.417859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.197663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.118408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.908390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.723569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.443811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.371852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.196681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.234609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.309984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.422805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.521113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.307668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.345058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.497883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.658205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.453505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.489897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.256824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.260800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.408505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.691807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.805713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.779937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.718700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.675091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.689440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.509326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.373964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.147680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.991573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.822525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.677805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.661539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.556631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.329434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.239194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.077382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.892574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.790069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.699324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.561130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.343643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.105251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.930904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.806736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.833139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.864780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.794057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.716774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.518598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.254734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.262286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.134829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.889760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.696419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.645479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.564300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.461557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.338748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.245733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.325202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.245289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.136457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.884428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.579327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.324050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.354946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.578078] LustreError: 172160:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5401.588052] LustreError: 172160:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5401.606565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.709762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.437624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.233400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.005720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.860950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.826753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.765727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.654781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.465893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.219514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.192936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.095831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.938184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.854846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.886937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.960664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.716610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.621115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.545931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.686849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.858942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.698786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.661208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.622636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.473157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.276197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.093082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.925634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.593868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.558490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.429884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.302442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.166834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.030916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.108791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.006415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.009320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.981574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.073627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.088282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.016549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.125821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.308164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.516775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.357803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.362184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.202707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.356145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.575988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.486898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.433389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.686744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.910933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.930711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.034736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.877519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.841552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.842238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.701559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.381709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.226951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.204362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.980377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.874811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.765526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.714947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.887045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.769066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.747928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.559542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.859494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.866207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.885935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.059970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.419220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.654293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.890804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.424115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.845490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.050980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.703330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.109095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.036916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.119047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.117892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.878203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.736502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.629129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.533357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.164471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.852901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.588309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.389079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.194082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.058664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.798375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.579550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.550521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.549827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.369782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.902079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.177631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.676231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.804440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.308873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.296304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.719098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.328672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.962785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.643340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.961842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.297415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.196219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.263164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.906500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.654729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.522536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.392367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.050456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.890623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.135665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.064812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.465171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.897727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.276759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.704748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.226665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.669727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.861507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.374577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.758748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.054964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.030775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.347606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.941102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.058315] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5543.068686] Lustre: Skipped 260 previous similar messages [ 5543.097484] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5543.116056] Lustre: Skipped 260 previous similar messages [ 5543.127827] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5543.134498] Lustre: Skipped 260 previous similar messages [ 5543.148742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.164842] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5543.171309] Lustre: Skipped 260 previous similar messages [ 5544.400032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.365708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.279394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.049439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.071637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.771892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.580551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.427595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.177509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.016754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.976222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.878614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.734729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.590539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.766815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.561864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.487750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.181515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.015856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.877697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.820569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.777626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.769448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.762768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.976576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.876785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.885240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.782523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.602498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.528734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.477255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.525217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.556219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.426736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.627061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.804739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.576545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.455408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.458500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.356261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.331398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.251027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.190757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.192617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.171432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.005237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.823060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.656697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.623821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.553903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.580429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.360774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.399981] LustreError: 172160:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5592.415197] LustreError: 172160:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5592.422375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.330205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.363192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.279706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.254996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.029781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.954890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.816661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.740499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.589969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.422968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.506506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.460397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.463222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.371803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.264808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.161873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.084832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.989534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.953460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.764602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.604399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.613291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.438238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.376627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.255605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.272877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.172576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.077596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.026648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.012446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.913233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.969634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.982305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.998949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.810720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.762144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.880026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.663365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.530351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.339175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.178231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.216055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.162354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.922900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.961819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.784573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.526040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.221983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.162744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.926645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.171492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.374789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.509844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.954597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.137954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.116431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.021341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.944326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.990135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.034036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.931712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.009991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.089156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.081413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.182056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.105608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.109867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.094549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.400968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.553040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.525162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.565207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.350460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.179293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.288604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.100075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.031527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.004775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.661187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.437631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.309356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.297535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.198717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.187628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.280365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.462073] LustreError: 172160:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5674.475928] LustreError: 172160:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5674.478119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.692650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.905906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.025760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.944190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.971319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.268492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.711811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.957725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.076570] LustreError: 172160:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5685.093158] LustreError: 172160:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5685.100025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.996118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.555805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.810875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.888887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.245309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.758516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.288163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.589982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.679712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.560155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.478260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.251706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.126248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.951609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.305347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.629188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.933750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.696363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.469523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.101222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.274205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.052238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.955947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.809517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.455719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.370275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.149768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.088141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.884375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.655797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.708391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.729450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.977539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.313884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.242446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.345084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.249613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.363244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.522097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.795116] LustreError: 172160:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5727.803751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.808994] LustreError: 172160:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5728.842394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.808733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.684355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.785985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.934062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.087714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.101445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.433532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.729338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.016510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.261603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.677602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.173595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.561370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.242329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.575746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.926871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.216683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.370642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.684635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.858080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.342714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.432873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.524159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.399639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.450761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.590841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.543044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.254808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.065168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.046483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.381254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.489520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.391023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.287390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.218374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.315588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.239563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.522313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.549432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.490983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.149930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.228474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.332426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.257527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.195684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.211573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.151537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.111692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.224623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.217609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.221713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.149732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.552374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.445068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.470567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.396632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.446829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.535465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.665723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.494564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.321367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.181726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.128541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.034746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.949752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.897368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.797660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.712431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.780526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.663644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.520072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.353868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.227932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.099886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.079815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.197314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.057983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.042981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.813461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.668553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.351568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.067943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.749801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.449505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.350658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.690560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.977871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.085116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.890620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.933780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.653451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.646042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.504783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.113668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.799723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.146291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.961423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.389233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.552034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.812505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.992869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.408882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.899370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.264542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.282396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.256370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.938419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.395205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.354885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.287235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.091938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.992263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.884532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.771973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.604418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.426868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.193507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.947321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.692371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.443723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.155045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.934556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.778620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.413248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.272469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.225476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.308525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.973279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.650807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.428243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.038595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.913344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.032305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.824118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.818569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.007914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.937827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.920627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.960856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.230569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.215699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.574777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.011370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.507950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.459079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.488962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.709398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.084953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.253096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.449592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.028031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.458368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.943401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.476043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.023649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.580674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.152834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.161282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.440277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.878735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.050632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.295696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.726068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.861352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.644152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.569730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.395776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.111791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.951827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.744144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.704706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.718234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.600104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.597171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.815474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.910995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.918599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.960658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.947694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.105585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.407743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.573671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.412902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.345972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.173141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.157681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.414985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.842694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.070799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.332078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.250421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.677650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.753665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.802157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.581958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.383477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.412275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.454129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.357249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.622854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.511561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.254101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.193440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.136334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.979576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.208852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.302099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.565500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.509511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.260192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.102642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.023982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.276864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.383620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.185849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.955573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.935584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.846628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.776524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.731985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.631027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.454472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.396944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.524305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.266867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.264754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.160879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.144916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.057419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.045853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.848593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.691427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.650349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.456083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.237016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.263374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.250719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.181034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.036423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.014314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.882175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.680073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.418627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.340292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.135712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.056302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.997593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.861760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.694136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.481925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.242753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.190317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.109389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.134860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.059965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.260510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.228571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.289228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.197549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.189874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.023251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.989847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.834692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.758438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.595329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.784267] LustreError: 174582:0:(ofd_dev.c:1794:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6463.803387] LustreError: 172160:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7184.882231] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 7184.891648] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7184.902725] LustreError: Skipped 1 previous similar message [ 7184.935229] Lustre: Skipped 454 previous similar messages [ 7184.959036] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7185.889186] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7185.904601] LustreError: Skipped 1 previous similar message [ 7185.912101] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7186.926281] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7186.941470] Lustre: Skipped 1 previous similar message [ 7190.611969] Lustre: server umount lustre-OST0000 complete [ 7196.643890] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7196.663146] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7200.651729] Lustre: server umount lustre-MDT0000 complete [ 7202.272048] LustreError: 172141:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7202.302976] LustreError: 172141:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 7204.690806] LustreError: 172123:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789136253 with bad export cookie 7800827583638983810 [ 7204.692714] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7204.703075] LustreError: 172123:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7205.039022] Lustre: server umount lustre-MDT0001 complete [ 7217.347665] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [ 7220.163173] Key type lgssc unregistered [ 7220.393941] LNet: 227435:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7220.403415] LNetError: 227435:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7220.419307] LNet: Removed LNI 192.168.206.143@tcp [ 7221.302953] Key type .llcrypt unregistered [ 7221.306341] Key type ._llcrypt unregistered [ 7243.844558] Key type ._llcrypt registered [ 7243.847660] Key type .llcrypt registered [ 7243.922225] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 7244.809444] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7244.832811] alg: No test for adler32 (adler32-zlib) [ 7245.873635] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 7246.171880] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [ 7247.879151] Key type lgssc registered [ 7248.987954] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7256.045863] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7256.076176] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7257.658186] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7262.156544] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 7267.812785] LustreError: 228843:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7267.826816] LustreError: 228843:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 7270.912315] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7271.360803] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7276.398158] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 7284.594711] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7290.780183] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7297.646547] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7298.050738] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7302.139663] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49664 to 0x280000401:49695) [ 7305.321825] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 7313.448113] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7321.150959] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7322.932231] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7322.947675] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7322.959383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7322.968601] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7322.969399] LustreError: 228863:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7325.319272] Lustre: lustre-OST0000: Not available for connect from 192.168.206.43@tcp (stopping) [ 7326.693957] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7326.698650] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7326.711399] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7327.718884] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7327.738911] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7327.749644] Lustre: Skipped 1 previous similar message [ 7330.405742] Lustre: lustre-OST0000: Not available for connect from 192.168.206.43@tcp (stopping) [ 7330.890607] Lustre: server umount lustre-OST0000 complete [ 7343.024074] LustreError: 228863:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7365.688449] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7365.944687] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7367.981144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7368.007079] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7373.073197] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 7377.292235] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7377.344488] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7382.611746] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7385.002095] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7393.817896] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff902482b5b800.ost_server_uuid 50 [ 7395.683128] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff902482b5b800.ost_server_uuid in FULL state after 0 sec [ 7400.937138] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7400.942487] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7400.958825] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7402.967807] Lustre: server umount lustre-OST0000 complete [ 7408.097490] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7408.111800] Lustre: Skipped 1 previous similar message [ 7413.143732] Lustre: server umount lustre-MDT0000 complete [ 7413.218061] LustreError: 228843:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7413.225622] LustreError: 228843:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 7416.406364] LustreError: 228823:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789136465 with bad export cookie 9697042075190795300 [ 7416.423935] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7416.424746] LustreError: 228823:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7427.248914] Lustre: server umount lustre-MDT0001 complete [ 7438.848778] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [ 7442.094484] Key type lgssc unregistered [ 7442.428530] LNet: 234056:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7442.438374] LNetError: 234056:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7442.458533] LNet: Removed LNI 192.168.206.143@tcp [ 7443.444339] Key type .llcrypt unregistered [ 7443.446284] Key type ._llcrypt unregistered [ 7483.404226] Key type ._llcrypt registered [ 7483.410234] Key type .llcrypt registered [ 7483.525916] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 7484.694829] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7484.736151] alg: No test for adler32 (adler32-zlib) [ 7485.751600] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 7485.995663] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [ 7487.751170] Key type lgssc registered [ 7489.017271] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7498.745154] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7498.760383] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7500.164135] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7504.648732] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 7510.497489] LustreError: 235786:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7510.508441] LustreError: 235786:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 7512.145610] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7512.462551] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7516.690809] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 7519.591227] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7525.965293] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7526.241765] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7531.859553] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 7539.169380] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7539.293976] Lustre: lustre-OST0001: new disk, initializing [ 7539.298144] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7539.303937] Lustre: 238094:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7539.397851] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7544.601770] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 7544.945509] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7544.956647] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7544.982974] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7546.041047] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49697 to 0x280000401:49727) [ 7555.303592] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7560.382163] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7571.938075] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7571.942342] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7571.952606] Lustre: Skipped 2 previous similar messages [ 7571.960719] Lustre: Skipped 3 previous similar messages [ 7577.059236] LustreError: 235787:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7577.070403] LustreError: 235787:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 7577.113045] Lustre: server umount lustre-MDT0000 complete [ 7579.393063] LustreError: 235767:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789136628 with bad export cookie 6801179989720331700 [ 7579.393611] LustreError: 236523:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 7579.394498] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7579.401201] LustreError: 235767:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7579.609209] Lustre: server umount lustre-MDT0001 complete [ 7591.488630] Lustre: server umount lustre-OST0000 complete [ 7604.703683] Lustre: server umount lustre-OST0001 complete [ 7621.509490] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 7629.470073] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7629.966513] LustreError: 241202:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7630.018046] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7633.802763] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 7635.427987] LustreError: 241202:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7635.441164] LustreError: 241202:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 7639.524277] LustreError: 241203:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7641.145321] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7641.645417] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7645.853215] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 7648.444638] Lustre: 242344:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7654.917133] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7660.716433] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 7666.540656] LustreError: 242698:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7666.569814] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 7668.186593] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7668.432771] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7668.438042] Lustre: Skipped 1 previous similar message [ 7673.469502] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 7673.845067] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 7674.869559] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49697 to 0x280000401:49759) [ 7679.746182] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7682.288663] Lustre: 244215:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7692.768491] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7692.778796] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7692.787629] Lustre: Skipped 1 previous similar message [ 7692.793034] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7695.329235] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7695.341102] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7695.354430] Lustre: Skipped 1 previous similar message [ 7698.756251] Lustre: server umount mylustre-MDT0000 complete [ 7700.458495] LustreError: 241202:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7700.480598] LustreError: 241202:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 7702.169645] LustreError: 241178:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789136751 with bad export cookie 6801179989720334563 [ 7702.180075] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7702.184331] LustreError: 241178:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7702.462239] Lustre: server umount mylustre-MDT0001 complete [ 7716.450648] Lustre: server umount mylustre-OST0000 complete [ 7729.890812] Lustre: server umount mylustre-OST0001 complete [ 7746.642826] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 7755.625892] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7756.199231] LustreError: 246522:0:(ldlm_lib.c:1228:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7756.267800] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7760.517785] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 7768.691976] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7773.107352] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 7775.372685] Lustre: 247663:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7780.968498] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7782.396256] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49697 to 0x280000401:49791) [ 7787.143694] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 7793.652107] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 7794.958340] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7800.314169] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:65) [ 7800.492432] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 7806.899818] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7810.109907] Lustre: 249534:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7820.257794] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7820.267100] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7820.280422] Lustre: Skipped 1 previous similar message [ 7820.290977] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7820.299415] Lustre: Skipped 3 previous similar messages [ 7822.329838] Lustre: server umount tfs-MDT0000 complete [ 7825.237875] LustreError: 246499:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789136874 with bad export cookie 6801179989720336740 [ 7825.243886] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7825.251935] LustreError: 246499:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7825.560455] Lustre: server umount tfs-MDT0001 complete [ 7839.015266] Lustre: server umount tfs-OST0000 complete [ 7841.248684] Lustre: 235050:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789136874/real 1789136874] req@ffff92c12605c780 x1876045630477824/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789136890 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7841.311666] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7841.336026] Lustre: Skipped 3 previous similar messages [ 7842.812728] Lustre: server umount tfs-OST0001 complete [ 7862.401260] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 7872.240535] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7872.668158] LustreError: 251788:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7872.692642] LustreError: 251788:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 7 previous similar messages [ 7872.763199] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7872.768214] Lustre: Skipped 3 previous similar messages [ 7877.898549] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 7885.868513] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7890.123702] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 7892.471555] Lustre: 252929:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7898.111439] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7904.253298] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 7911.676728] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 7912.418555] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7917.826622] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49697 to 0x280000401:49823) [ 7917.833523] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:97) [ 7918.171550] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 7924.018533] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7926.928832] Lustre: 254802:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7932.499599] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 10:29:38 (1789136978) [ 7947.745562] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7947.756447] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7947.768465] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7947.775504] Lustre: Skipped 3 previous similar messages [ 7953.486413] Lustre: server umount lustre-MDT0000 complete [ 7956.328944] LustreError: 251765:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789137005 with bad export cookie 6801179989720338658 [ 7956.334880] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7956.342307] LustreError: 251765:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7959.010587] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7959.017866] Lustre: Skipped 5 previous similar messages [ 7962.682987] Lustre: server umount lustre-MDT0001 complete [ 7971.787898] Lustre: server umount lustre-OST0000 complete [ 7981.410751] Lustre: server umount lustre-OST0001 complete [ 7988.147921] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_hostid [ 7997.375743] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 8033.809369] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8034.069144] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8034.103531] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8034.209239] Lustre: lustre-MDT0000: new disk, initializing [ 8034.290947] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8034.297178] Lustre: Skipped 3 previous similar messages [ 8034.306089] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8038.728935] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 8047.805829] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8047.929478] Lustre: 257985:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8047.995699] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8047.998846] Lustre: Skipped 1 previous similar message [ 8048.087623] Lustre: lustre-MDT0001: new disk, initializing [ 8048.278577] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8048.297936] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8052.045588] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 8061.932094] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8062.296562] Lustre: lustre-OST0000: new disk, initializing [ 8062.301179] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8062.310828] Lustre: 259567:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8063.792082] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8063.809982] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8063.948477] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8068.216656] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 8077.076236] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8077.231057] Lustre: lustre-OST0001: new disk, initializing [ 8077.235350] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8077.240874] Lustre: 260438:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8078.432472] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8078.441116] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8078.533135] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8083.263175] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 8094.745789] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 10:32:20 (1789137140) [ 8109.017633] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 10:32:34 (1789137154) [ 8124.917173] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8124.918920] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8124.926283] Lustre: Skipped 6 previous similar messages [ 8124.947503] Lustre: Skipped 3 previous similar messages [ 8138.207110] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8138.282398] Lustre: server umount lustre-OST0000 complete [ 8140.266541] LustreError: 259550:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8140.293576] LustreError: 259550:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 12 previous similar messages [ 8145.392642] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8148.254431] Lustre: server umount lustre-MDT0000 complete [ 8151.392621] LustreError: 257977:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789137200 with bad export cookie 6801179989720340436 [ 8151.400209] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8151.407027] LustreError: 257977:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8157.838322] Lustre: server umount lustre-MDT0001 complete [ 8167.131729] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [ 8236.323540] Lustre: server umount lustre-OST0001 complete [ 8241.851390] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_hostid [ 8258.052873] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 8291.859435] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8292.093305] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8292.114344] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8292.187643] Lustre: lustre-MDT0000: new disk, initializing [ 8292.310758] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8292.323779] Lustre: Skipped 3 previous similar messages [ 8292.354688] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8297.320404] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 8308.220273] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8308.357258] Lustre: 265290:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8308.439151] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8308.443963] Lustre: Skipped 1 previous similar message [ 8308.551758] Lustre: lustre-MDT0001: new disk, initializing [ 8308.836528] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8308.845508] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8313.787476] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 8323.622356] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8328.622713] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8334.571695] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8334.812387] Lustre: 267258:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8336.022559] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8336.089904] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8339.756351] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 8348.064861] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8356.836117] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8356.849327] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8356.852973] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8356.863525] Lustre: Skipped 5 previous similar messages [ 8356.873610] Lustre: Skipped 10 previous similar messages [ 8359.838758] Lustre: server umount lustre-OST0000 complete [ 8365.024636] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8365.034805] LustreError: Skipped 1 previous similar message [ 8369.411278] Lustre: server umount lustre-MDT0000 complete [ 8372.529412] LustreError: 265282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789137421 with bad export cookie 6801179989720341906 [ 8372.533839] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8372.542083] LustreError: 265282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8372.892767] Lustre: server umount lustre-MDT0001 complete [ 8382.480713] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [ 8385.349671] Key type lgssc unregistered [ 8385.646182] LNet: 269018:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8385.652261] LNetError: 269018:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8385.672578] LNet: Removed LNI 192.168.206.143@tcp [ 8386.560162] Key type .llcrypt unregistered [ 8386.564058] Key type ._llcrypt unregistered [ 8408.829449] Key type ._llcrypt registered [ 8408.831401] Key type .llcrypt registered [ 8408.928269] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 8410.071696] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8410.091567] alg: No test for adler32 (adler32-zlib) [ 8411.220701] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 8411.418977] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [ 8413.135235] Key type lgssc registered [ 8414.159714] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8420.299570] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8420.320339] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8421.730095] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8425.930965] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 8432.101917] LustreError: 270481:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8432.126479] LustreError: 270481:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 8436.426057] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8436.776732] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8440.985491] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 8448.617520] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8453.545856] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8459.299944] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8459.531820] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8464.654223] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 8467.758618] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8471.759926] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8472.783712] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8477.154332] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8477.161370] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8477.171726] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8478.179515] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8478.182530] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8478.194787] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8480.852054] Lustre: server umount lustre-OST0000 complete [ 8487.913612] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8487.926443] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8487.938489] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8487.942607] Lustre: Skipped 1 previous similar message [ 8491.290747] Lustre: server umount lustre-MDT0000 complete [ 8493.544258] LustreError: 270477:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8493.593818] LustreError: 270477:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 8494.996778] LustreError: 270464:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789137543 with bad export cookie 16368953280825182373 [ 8495.001215] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8495.025308] LustreError: 270464:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8495.530253] Lustre: server umount lustre-MDT0001 complete [ 8507.433474] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [ 8510.372591] Key type lgssc unregistered [ 8510.738082] LNet: 274078:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8510.742854] LNetError: 274078:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8510.756186] LNet: Removed LNI 192.168.206.143@tcp [ 8511.672382] Key type .llcrypt unregistered [ 8511.675333] Key type ._llcrypt unregistered [ 8527.887495] Key type ._llcrypt registered [ 8527.890648] Key type .llcrypt registered [ 8528.048381] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 8529.181903] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8529.235467] alg: No test for adler32 (adler32-zlib) [ 8530.322192] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 8530.559598] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [ 8532.271158] Key type lgssc registered [ 8533.281823] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8541.165483] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8541.185259] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8556.767463] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8561.887330] LustreError: 275271:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.143@tcp: failed processing log, type 4: rc = -110 [ 8588.639176] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8593.222367] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 8599.614928] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8601.418713] Lustre: server umount lustre-OST0000 complete [ 8615.271724] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [ 8617.348302] Key type lgssc unregistered [ 8617.599420] LNet: 276904:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8617.610200] LNetError: 276904:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8617.627292] LNet: Removed LNI 192.168.206.143@tcp [ 8618.425316] Key type .llcrypt unregistered [ 8618.429807] Key type ._llcrypt unregistered [ 8652.291577] Key type ._llcrypt registered [ 8652.293874] Key type .llcrypt registered [ 8652.373964] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 8653.275342] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8653.453683] alg: No test for adler32 (adler32-zlib) [ 8654.666668] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 8655.043622] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [ 8656.775144] Key type lgssc registered [ 8657.887256] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8670.744993] loop0: detected capacity change from 0 to 204800000 [ 8670.943611] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8670.958886] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8671.278628] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8672.439539] loop0: detected capacity change from 0 to 204800000 [ 8672.661897] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8672.671799] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8673.195806] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8674.233839] loop0: detected capacity change from 0 to 204800000 [ 8674.409345] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8674.417711] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8674.574227] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8675.821417] loop0: detected capacity change from 0 to 204800000 [ 8676.136654] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8676.152300] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8676.396447] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8677.519079] loop0: detected capacity change from 0 to 204800000 [ 8677.551864] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8681.615360] loop0: detected capacity change from 0 to 204800000 [ 8681.650731] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8685.216934] loop0: detected capacity change from 0 to 204800000 [ 8685.242663] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8689.224302] loop0: detected capacity change from 0 to 204800000 [ 8689.250246] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8693.514150] loop0: detected capacity change from 0 to 204800000 [ 8693.631880] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8693.691642] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8693.756510] Lustre: 280317:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8693.765316] Lustre: 280317:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8693.772680] Lustre: 280317:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8693.778985] Lustre: 280317:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8693.787729] Lustre: 280317:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8693.795287] Lustre: 280317:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8693.802979] Lustre: 280317:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8693.812498] Lustre: 280317:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8693.821159] Lustre: 280317:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8693.830151] Lustre: 280317:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8693.837434] Lustre: 280317:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8693.856439] Lustre: 280317:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8693.866351] Lustre: 280317:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8693.881284] Lustre: 280317:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8699.481683] Lustre: server umount MGS complete [ 8700.394149] loop0: detected capacity change from 0 to 204800000 [ 8700.510625] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8700.979314] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8701.045665] Lustre: lustre-MDT0000: Aborting client recovery [ 8702.137441] loop1: detected capacity change from 0 to 204800000 [ 8702.339429] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8702.432318] Lustre: 280748:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8702.446487] Lustre: 280748:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8702.467838] Lustre: 280748:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8702.487548] Lustre: 280748:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8702.509898] Lustre: 280748:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8702.521367] Lustre: 280748:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8702.536552] Lustre: 280748:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8702.563955] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8702.845201] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8702.905280] Lustre: lustre-MDT0001: Aborting client recovery [ 8703.989260] loop2: detected capacity change from 0 to 204800000 [ 8704.105626] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8704.178235] Lustre: 280827:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8704.192254] Lustre: 280827:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8704.207256] Lustre: 280827:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8704.219491] Lustre: 280827:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8704.232435] Lustre: 280827:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8704.245610] Lustre: 280827:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8704.256723] Lustre: 280827:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8704.376552] Lustre: Found index 0 for lustre-OST0000, updating log [ 8704.452126] Lustre: 280846:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8704.535373] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8704.565156] Lustre: lustre-OST0000: Aborting recovery [ 8705.499401] loop3: detected capacity change from 0 to 204800000 [ 8705.649158] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8705.714962] Lustre: 280911:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8705.723242] Lustre: 280911:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8705.752120] Lustre: 280911:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8705.767542] Lustre: 280911:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8705.783581] Lustre: 280911:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8705.803819] Lustre: 280911:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8705.820516] Lustre: 280911:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8705.850916] Lustre: Found index 1 for lustre-OST0001, updating log [ 8705.877331] Lustre: 280917:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8705.973625] Lustre: lustre-OST0001: Aborting recovery [ 8711.087231] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8711.092660] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8711.100552] LustreError: 280692:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 11, retries 0, failed: rc = -108 [ 8711.149108] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8711.160571] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8712.100281] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8712.103324] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8714.173802] LustreError: 280659:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 7860us: rc = -2 [ 8714.692367] LustreError: 280658:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 2307us: rc = -2 [ 8714.719915] LustreError: 280658:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 20 previous similar messages [ 8727.520209] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8727.523149] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8727.540232] Lustre: Skipped 2 previous similar messages [ 8727.560372] Lustre: Skipped 3 previous similar messages [ 8732.644840] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8732.656280] Lustre: Skipped 3 previous similar messages [ 8733.609915] Lustre: server umount lustre-MDT0000 complete [ 8734.709875] LustreError: 280642:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789137783 with bad export cookie 6820039872099990482 [ 8734.713431] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8734.721879] LustreError: 280642:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8737.766501] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8737.768827] LustreError: 281177:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8737.773774] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8737.779510] Lustre: Skipped 2 previous similar messages [ 8737.798890] LustreError: 281177:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 8739.808632] LustreError: 280657:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8739.816302] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8739.829790] LustreError: 280657:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 8739.843341] Lustre: Skipped 2 previous similar messages [ 8741.214890] Lustre: server umount lustre-MDT0001 complete [ 8748.441772] Lustre: server umount lustre-OST0000 complete [ 8755.782755] Lustre: server umount lustre-OST0001 complete [ 8764.994360] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 10:43:30 (1789137810) [ 8795.956324] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_hostid [ 8804.855918] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 8835.418927] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8835.625109] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8835.648940] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8835.717311] Lustre: lustre-MDT0000: new disk, initializing [ 8835.786059] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8835.790177] Lustre: Skipped 1 previous similar message [ 8835.804502] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8839.664346] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 8850.317609] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8850.451890] Lustre: 284987:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8850.526156] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8850.529766] Lustre: Skipped 1 previous similar message [ 8850.590556] Lustre: lustre-MDT0001: new disk, initializing [ 8850.701469] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8850.756232] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8850.780494] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8857.182404] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 8869.167500] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8875.370356] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8882.045958] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8882.301347] Lustre: lustre-OST0000: new disk, initializing [ 8882.306483] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8882.315627] Lustre: 286957:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8882.385555] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8888.721317] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 8889.910443] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8889.928319] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8890.102880] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8897.473803] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8902.271782] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8911.053573] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8918.063330] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8962.021205] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8962.021840] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8962.033276] LustreError: Skipped 1 previous similar message [ 8962.047139] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8962.064159] Lustre: Skipped 1 previous similar message [ 8965.608518] Lustre: server umount lustre-OST0000 complete [ 8972.261778] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8972.275386] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8972.277202] Lustre: Skipped 2 previous similar messages [ 8972.301066] Lustre: Skipped 1 previous similar message [ 8975.142521] Lustre: server umount lustre-MDT0000 complete [ 8977.376626] LustreError: 286356:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8977.385877] LustreError: 286356:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 8978.343333] LustreError: 286956:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789138027 with bad export cookie 6820039872099993184 [ 8978.345075] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8978.364161] LustreError: 286956:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8978.645651] Lustre: server umount lustre-MDT0001 complete [ 8985.946557] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8989.296356] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 8993.215227] Lustre: server umount MGS complete [ 9003.315156] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9003.652936] LustreError: 290816:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9003.709661] LustreError: 290829:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9003.780629] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9007.594157] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 9009.120964] LustreError: 290817:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9015.312544] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9018.715879] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 9024.377341] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9029.382863] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9035.348434] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9040.809222] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 9046.457523] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9080.809557] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9080.809565] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9080.813668] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9080.816066] Lustre: Skipped 1 previous similar message [ 9080.838407] Lustre: Skipped 1 previous similar message [ 9083.975788] Lustre: server umount lustre-OST0000 complete [ 9087.462467] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9091.041140] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9091.050074] Lustre: Skipped 1 previous similar message [ 9101.279214] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9101.409842] Lustre: server umount lustre-MDT0000 complete [ 9104.271036] LustreError: 290797:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789138153 with bad export cookie 6820039872099995025 [ 9104.275091] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9104.279228] LustreError: 290797:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9104.527906] Lustre: server umount lustre-MDT0001 complete [ 9113.116387] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [ 9115.217512] Key type lgssc unregistered [ 9115.490906] LNet: 294770:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9115.497073] LNetError: 294770:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9115.512663] LNet: Removed LNI 192.168.206.143@tcp [ 9116.221564] Key type .llcrypt unregistered [ 9116.224067] Key type ._llcrypt unregistered [ 9145.125551] Key type ._llcrypt registered [ 9145.128532] Key type .llcrypt registered [ 9145.222552] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_hostid [ 9156.477704] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 9157.276609] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9157.297163] alg: No test for adler32 (adler32-zlib) [ 9158.264186] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 9158.453667] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [ 9160.088284] Key type lgssc registered [ 9160.979717] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9191.893470] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9191.909396] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9193.113673] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9193.147939] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9193.229321] Lustre: lustre-MDT0000: new disk, initializing [ 9193.298096] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9193.318248] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9197.080263] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 9206.008845] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9206.144714] Lustre: 298719:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9206.207107] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9206.211828] Lustre: Skipped 1 previous similar message [ 9206.297540] Lustre: lustre-MDT0001: new disk, initializing [ 9206.394334] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9206.426622] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9206.444283] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9210.237268] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 9220.837635] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9228.105075] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9234.466963] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9234.682368] Lustre: lustre-OST0000: new disk, initializing [ 9234.686668] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9234.691053] Lustre: 300688:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9234.747938] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9240.650947] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 9242.721115] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9242.727353] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9242.833443] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9248.955817] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9255.189400] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9265.604363] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9271.728820] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9277.348359] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9319.915806] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9319.938508] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9319.942425] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9325.026945] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9325.036302] Lustre: Skipped 2 previous similar messages [ 9325.320723] Lustre: server umount lustre-OST0000 complete [ 9330.147193] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9330.161530] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9330.178172] Lustre: Skipped 2 previous similar messages [ 9330.206275] Lustre: Skipped 1 previous similar message [ 9335.266791] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9335.287628] Lustre: Skipped 1 previous similar message [ 9335.646493] Lustre: server umount lustre-MDT0000 complete [ 9338.626953] LustreError: 298711:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789138387 with bad export cookie 1720383777914920827 [ 9338.628196] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9338.638160] LustreError: 298711:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9338.973529] Lustre: server umount lustre-MDT0001 complete [ 9348.460671] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9352.271393] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 9356.406112] Lustre: server umount MGS complete [ 9367.101226] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9367.477187] LustreError: 304571:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9367.544570] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9372.375305] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 9372.641752] LustreError: 304558:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9372.651331] LustreError: 304558:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 9377.762294] LustreError: 304559:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9380.816279] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9381.318441] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9385.360581] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 9392.029318] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9396.740656] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9402.006147] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9402.275063] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9407.240528] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 9413.531489] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9442.274942] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9442.285431] LustreError: Skipped 1 previous similar message [ 9442.288162] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9442.298608] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9442.737306] Lustre: server umount lustre-OST0000 complete [ 9447.904662] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9447.912060] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9453.536748] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9453.541356] Lustre: Skipped 4 previous similar messages [ 9460.191132] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9460.333032] Lustre: server umount lustre-MDT0000 complete [ 9463.778391] LustreError: 304559:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9463.794025] LustreError: 304559:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 9464.181125] LustreError: 304539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789138513 with bad export cookie 1720383777914922752 [ 9464.182555] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9464.196238] LustreError: 304539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9464.375841] Lustre: server umount lustre-MDT0001 complete [ 9475.888649] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [ 9478.639436] Key type lgssc unregistered [ 9478.859823] LNet: 308509:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9478.877418] LNetError: 308509:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9478.891449] LNet: Removed LNI 192.168.206.143@tcp [ 9479.585378] Key type .llcrypt unregistered [ 9479.587372] Key type ._llcrypt unregistered [ 9499.355292] Key type ._llcrypt registered [ 9499.357240] Key type .llcrypt registered [ 9499.439226] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_hostid [ 9511.753453] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 9513.130222] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9513.177996] alg: No test for adler32 (adler32-zlib) [ 9514.179651] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 9514.369569] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [ 9516.039537] Key type lgssc registered [ 9517.165428] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9541.315943] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [ 9556.943341] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9557.029936] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9558.563820] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9558.611921] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9558.782139] Lustre: lustre-MDT0000: new disk, initializing [ 9558.980030] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9559.010448] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9563.680253] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 9574.157881] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9574.389742] Lustre: lustre-OST0000: new disk, initializing [ 9574.392833] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9574.396773] Lustre: Skipped 1 previous similar message [ 9574.401523] Lustre: 312536:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9574.453876] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9579.710505] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [ 9582.717665] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9582.724918] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9582.750471] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10473.999711] Lustre: server umount lustre-MDT0000 complete [10477.319088] LustreError: 311723:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789139526 with bad export cookie 6032605034118288993 [10477.327659] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10487.911564] Lustre: server umount lustre-OST0000 complete [10511.020574] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_hostid [10521.760712] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [10561.071840] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10561.299398] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10561.324805] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10561.415390] Lustre: lustre-MDT0000: new disk, initializing [10561.486904] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10561.499308] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10565.819064] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [10575.413495] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10575.507857] Lustre: 316889:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10575.551542] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10575.555542] Lustre: Skipped 1 previous similar message [10575.610161] Lustre: lustre-MDT0001: new disk, initializing [10575.677587] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10575.703833] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10575.712950] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10579.322483] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [10587.491410] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10592.639323] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10598.362038] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10598.616739] Lustre: lustre-OST0000: new disk, initializing [10598.622409] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10598.637139] Lustre: 318857:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10598.710226] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10603.632433] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [10607.135112] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10607.143270] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10607.243533] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10612.202913] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10616.584630] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10616.844182] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10621.838976] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10622.039787] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10627.553642] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10627.565719] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10627.580925] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10628.066218] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10628.070926] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10628.088844] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10628.092454] Lustre: Skipped 1 previous similar message [10629.846721] Lustre: server umount lustre-OST0000 complete [10633.188352] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10633.191452] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10633.212041] Lustre: Skipped 1 previous similar message [10633.225780] Lustre: Skipped 1 previous similar message [10638.305081] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10638.310850] Lustre: Skipped 1 previous similar message [10639.361573] Lustre: server umount lustre-MDT0000 complete [10642.807915] LustreError: 316881:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789139691 with bad export cookie 6032605034121229812 [10642.813932] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10642.820989] LustreError: 316881:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10643.180895] Lustre: server umount lustre-MDT0001 complete [10646.310973] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10647.962527] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 11:14:54 (1789139694) [10653.657372] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10654.086047] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10657.555066] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [10659.300816] LustreError: 321169:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10659.328027] LustreError: 321169:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [10663.394618] LustreError: 321170:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10664.803620] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10665.134258] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10668.902844] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [10676.108260] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10681.482258] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10687.966484] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10688.231257] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10693.620097] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [10703.230346] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10716.266884] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10716.640526] Lustre: lustre-OST0001: new disk, initializing [10716.647623] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10716.657648] Lustre: 324006:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10716.807675] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10721.908259] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10721.921196] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10721.976506] LustreError: 324199:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10721.987212] LustreError: 324199:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10723.041305] LustreError: 324199:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10724.083318] LustreError: 324199:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10724.097355] LustreError: 324199:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10724.794568] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [10726.121900] LustreError: 324207:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10726.134627] LustreError: 324207:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10730.287756] LustreError: 324207:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10730.302550] LustreError: 324207:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [10735.312820] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10735.565212] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10738.543239] LustreError: 324199:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10738.552363] LustreError: 324199:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [10741.217098] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff902482b5d800.ost_server_uuid 50 [10742.715635] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff902482b5d800.ost_server_uuid in FULL state after 0 sec [10748.356447] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff902482b5d800.ost_server_uuid 50 [10749.790842] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff902482b5d800.ost_server_uuid in FULL state after 0 sec [10751.948505] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10765.937323] Lustre: lustre-OST0001: Not available for connect from 192.168.206.43@tcp (stopping) [10767.328847] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10767.339855] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10776.159210] Lustre: lustre-OST0001: Not available for connect from 192.168.206.43@tcp (stopping) [10776.173606] Lustre: Skipped 7 previous similar messages [10779.103444] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10779.266097] Lustre: server umount lustre-OST0001 complete [10781.281330] LustreError: 323920:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.206.43@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10783.726739] LustreError: 325543:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10783.734586] LustreError: 325543:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [10785.763337] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10785.776170] LustreError: Skipped 1 previous similar message [10785.783424] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10785.793712] Lustre: Skipped 1 previous similar message [10788.840596] LustreError: 322988:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10788.861156] LustreError: 322988:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [10791.476278] Lustre: server umount lustre-OST0000 complete [10798.569956] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10798.583277] LustreError: Skipped 1 previous similar message [10798.587835] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10798.605257] Lustre: Skipped 1 previous similar message [10798.610465] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10798.623911] Lustre: Skipped 6 previous similar messages [10809.311118] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10809.494724] Lustre: server umount lustre-MDT0000 complete [10813.409173] LustreError: 325659:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10813.435256] LustreError: 325659:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [10813.667588] LustreError: 321152:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789139862 with bad export cookie 6032605034121230694 [10813.682948] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10814.006119] Lustre: server umount lustre-MDT0001 complete [10825.579431] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [10828.464307] Key type lgssc unregistered [10828.758886] LNet: 326703:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10828.767492] LNetError: 326703:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10828.779897] LNet: Removed LNI 192.168.206.143@tcp [10829.629511] Key type .llcrypt unregistered [10829.632253] Key type ._llcrypt unregistered [10853.132048] Key type ._llcrypt registered [10853.134652] Key type .llcrypt registered [10853.246646] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [10854.126497] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10854.182076] alg: No test for adler32 (adler32-zlib) [10855.178784] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [10855.352025] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [10856.999152] Key type lgssc registered [10857.883636] Lustre: Echo OBD driver; http://www.lustre.org/ [10864.095384] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10864.111905] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10865.533648] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10870.025623] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [10875.872814] LustreError: 328120:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10875.882484] LustreError: 328120:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [10876.963134] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10880.996088] LustreError: 328119:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10883.684888] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10884.191379] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10889.036779] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [10898.301818] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10898.614273] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10904.609747] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [10909.859640] LustreError: 329800:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10910.926668] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10911.494524] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10916.322069] LustreError: 329801:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10916.332492] LustreError: 329801:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [10917.536378] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10917.729167] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10922.997718] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10923.258322] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [10931.331450] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10939.168780] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10939.506779] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11045.857269] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11045.883897] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11045.892516] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11045.903250] Lustre: Skipped 1 previous similar message [11049.663510] Lustre: server umount lustre-OST0001 complete [11050.982239] LustreError: 329802:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11051.000058] LustreError: 329802:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [11056.101645] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11056.119562] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11056.126184] Lustre: Skipped 2 previous similar messages [11061.218077] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11061.221020] LustreError: 331129:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11061.221620] Lustre: Skipped 1 previous similar message [11061.241315] LustreError: 331129:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [11066.337491] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11069.407176] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11069.571831] Lustre: server umount lustre-OST0000 complete [11073.506438] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11073.518361] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11073.528849] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11073.537218] Lustre: Skipped 1 previous similar message [11076.580087] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11081.697166] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11081.704066] Lustre: Skipped 3 previous similar messages [11087.327873] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11087.515398] Lustre: server umount lustre-MDT0000 complete [11090.947556] LustreError: 328101:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789140139 with bad export cookie 10305904392201600571 [11090.963807] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11091.325592] Lustre: server umount lustre-MDT0001 complete [11103.459671] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [11106.339583] Key type lgssc unregistered [11106.572609] LNet: 333073:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11106.585790] LNetError: 333073:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11106.641565] LNet: Removed LNI 192.168.206.143@tcp [11107.470786] Key type .llcrypt unregistered [11107.473986] Key type ._llcrypt unregistered [11151.189163] Key type ._llcrypt registered [11151.191342] Key type .llcrypt registered [11151.282514] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [11152.328242] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11152.350278] alg: No test for adler32 (adler32-zlib) [11153.353472] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [11153.589748] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [11155.248565] Key type lgssc registered [11156.047212] Lustre: Echo OBD driver; http://www.lustre.org/ [11164.032685] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11164.044697] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11165.499739] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11169.264944] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [11175.907051] LustreError: 335362:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11175.933466] LustreError: 335362:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [11176.767019] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11177.039461] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11180.708761] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [11183.352514] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11188.707539] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11188.977670] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11191.015991] LustreError: 336853:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11191.031283] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11193.232831] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [11196.389403] LustreError: 337035:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11199.751532] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11199.970874] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11202.039904] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11204.820633] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [11212.704492] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11218.146432] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11232.748875] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11232.754860] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11232.764414] Lustre: Skipped 2 previous similar messages [11232.773582] Lustre: Skipped 3 previous similar messages [11234.466795] Lustre: server umount lustre-MDT0000 complete [11237.776053] LustreError: 335343:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789140286 with bad export cookie 2042782784403135614 [11237.783046] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11237.787470] LustreError: 335343:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11237.857495] LustreError: 335357:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11237.862483] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11237.871199] LustreError: 335357:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [11237.894444] Lustre: Skipped 1 previous similar message [11237.900516] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11237.909143] Lustre: Skipped 1 previous similar message [11238.108319] Lustre: server umount lustre-MDT0001 complete [11247.562791] Lustre: server umount lustre-OST0000 complete [11250.817466] Lustre: server umount lustre-OST0001 complete [11266.955802] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [11279.044448] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11279.510673] LustreError: 340626:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11279.575346] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11283.780965] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [11290.081102] LustreError: 340627:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11290.100608] LustreError: 340627:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [11291.220240] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11291.592253] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11294.749293] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [11297.091243] Lustre: 341766:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11302.562033] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11304.953416] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11309.393607] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [11311.075486] LustreError: 342122:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11311.108332] LustreError: 342122:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [11318.365051] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11318.793699] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11318.799800] Lustre: Skipped 1 previous similar message [11323.891871] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11324.883428] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [11333.889809] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11338.174203] Lustre: 343639:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11342.824973] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11342.834477] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11342.861935] Lustre: Skipped 1 previous similar message [11342.871405] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11349.473634] LustreError: 340627:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11349.494147] LustreError: 340627:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 5 previous similar messages [11349.714686] Lustre: server umount lustre-MDT0000 complete [11352.452618] LustreError: 340608:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789140401 with bad export cookie 2042782784403137315 [11352.453926] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11352.461802] LustreError: 340608:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11352.722515] Lustre: server umount lustre-MDT0001 complete [11365.193237] Lustre: server umount lustre-OST0000 complete [11379.156771] Lustre: server umount lustre-OST0001 complete [11382.905545] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11384.577197] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11386.236737] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 11:27:12 (1789140432) [11406.969688] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [11415.147665] loop0: detected capacity change from 0 to 1099511627776 [11415.718573] SGI XFS with ACLs, security attributes, quota, no debug enabled [11415.724634] *** VALIDATE xfs *** [11415.735968] XFS (loop0): Mounting V5 Filesystem [11415.772129] XFS (loop0): Ending clean mount [11417.533573] loop1: detected capacity change from 0 to 18691697672192 [11504.600798] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_hostid [11514.469100] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [11551.841420] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11551.983326] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11552.012804] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11552.071858] Lustre: lustre-MDT0000: new disk, initializing [11552.138449] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11552.163207] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11557.061884] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [11566.428359] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11566.558142] Lustre: 348772:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11566.606833] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11566.609142] Lustre: Skipped 1 previous similar message [11566.659037] Lustre: lustre-MDT0001: new disk, initializing [11566.821589] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11566.847674] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11570.136753] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [11578.924980] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11583.664142] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11589.050101] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11589.290484] Lustre: lustre-OST0000: new disk, initializing [11589.293759] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11589.298890] Lustre: 350743:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11590.841733] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11590.848071] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11590.938938] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11594.362653] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [11602.813834] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11607.254244] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11607.420448] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11611.674201] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11611.841924] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11616.741232] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11616.742456] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11616.746531] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11616.746539] Lustre: Skipped 4 previous similar messages [11616.752653] LustreError: Skipped 1 previous similar message [11616.777223] Lustre: Skipped 4 previous similar messages [11619.862691] Lustre: server umount lustre-OST0000 complete [11623.403134] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11623.412589] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11623.425848] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11623.433543] Lustre: Skipped 1 previous similar message [11629.567948] Lustre: server umount lustre-MDT0000 complete [11632.096349] LustreError: 348785:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11632.131534] LustreError: 348785:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [11632.871418] LustreError: 350742:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789140681 with bad export cookie 2042782784403138869 [11632.876714] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11632.884254] LustreError: 350742:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11633.196952] Lustre: server umount lustre-MDT0001 complete [11639.958629] XFS (loop0): Unmounting Filesystem [11643.846656] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 11:31:29 (1789140689) [11650.683568] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11651.136616] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11651.143031] Lustre: Skipped 2 previous similar messages [11655.169613] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [11663.996577] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11668.931543] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [11676.042467] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11682.432777] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11688.705861] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11696.100109] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [11704.281178] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11716.070154] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11716.070790] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11716.083077] LustreError: Skipped 1 previous similar message [11716.084604] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11716.084611] Lustre: Skipped 2 previous similar messages [11716.109147] Lustre: Skipped 2 previous similar messages [11719.533691] Lustre: server umount lustre-OST0000 complete [11725.793191] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11736.551876] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11736.556324] Lustre: Skipped 6 previous similar messages [11738.079192] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11738.250825] Lustre: server umount lustre-MDT0000 complete [11743.458506] LustreError: 353281:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789140792 with bad export cookie 2042782784403139751 [11743.462052] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11743.464446] LustreError: 353281:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11744.113950] Lustre: server umount lustre-MDT0001 complete [11756.169043] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [11758.765670] Key type lgssc unregistered [11759.015964] LNet: 356934:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11759.021795] LNetError: 356934:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11759.045494] LNet: Removed LNI 192.168.206.143@tcp [11759.776151] Key type .llcrypt unregistered [11759.777969] Key type ._llcrypt unregistered [11784.470521] Key type ._llcrypt registered [11784.472788] Key type .llcrypt registered [11784.598750] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [11785.601230] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11785.618611] alg: No test for adler32 (adler32-zlib) [11786.620977] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [11786.848108] LNet: Added LNI 192.168.206.143@tcp [8/256/0/180] [11788.495112] Key type lgssc registered [11789.441721] Lustre: Echo OBD driver; http://www.lustre.org/ [11796.483367] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11796.500259] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11797.944758] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11802.148962] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [11807.205904] LustreError: 358504:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11807.218752] LustreError: 358504:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [11809.832045] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11810.194855] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11814.459666] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [11822.079945] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11827.699806] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11835.014897] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11835.257827] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11841.184224] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [11850.157210] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11861.478143] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11861.505344] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11861.535794] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11863.015735] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11863.021462] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11863.033233] Lustre: Skipped 1 previous similar message [11865.287561] Lustre: server umount lustre-MDT0001 complete [11868.138039] LustreError: 359541:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11868.159815] LustreError: 359541:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [11873.255502] LustreError: 358505:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11873.266088] LustreError: 358505:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [11874.544724] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11874.681607] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11874.689515] Lustre: Found index 1 for lustre-MDT0001, updating log [11874.757926] Lustre: 358493:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11875.154034] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11878.976984] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [11880.439782] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11881.484837] LustreError: 362078:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11881.495424] LustreError: 362078:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11881.501656] LustreError: 358496:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11881.569916] LustreError: 362085:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.143@tcp: cfg command failed: rc = -17 [11881.582986] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11881.582986] [11888.763697] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.43@tcp (stopping) [11888.768769] Lustre: Skipped 1 previous similar message [11890.670107] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11890.672583] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11892.893137] Lustre: server umount lustre-MDT0001 complete [11893.857189] LustreError: 358498:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.43@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11893.886661] LustreError: 358498:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [11898.973049] LustreError: 358498:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.43@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11899.003450] LustreError: 358498:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [11902.336884] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11902.463244] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11902.468701] Lustre: Found index 1 for lustre-MDT0001, updating log [11902.513322] Lustre: 358494:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11902.842478] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11907.015948] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [11908.078461] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11911.155889] LustreError: 363189:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11911.160600] LustreError: 363189:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11911.169880] LustreError: 358496:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11911.180326] LustreError: 358496:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11911.278854] LustreError: 363192:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11911.289485] LustreError: 363192:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11911.296925] LustreError: 363192:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.143@tcp: cfg command failed: rc = -17 [11911.304111] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.206.143@tcp [11911.304111] [12215.289336] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12215.291292] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12215.301207] Lustre: Skipped 1 previous similar message [12215.313950] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12215.353967] Lustre: Skipped 3 previous similar messages [12219.848406] Lustre: server umount lustre-MDT0001 complete [12220.385270] LustreError: 358504:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12220.408437] LustreError: 358504:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [12231.980434] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12232.066603] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12232.070923] Lustre: Found index 1 for lustre-MDT0001, updating log [12232.131789] Lustre: 358494:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12232.472395] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12237.213502] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [12237.815581] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12238.858663] LustreError: 371682:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12238.869357] LustreError: 371682:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12238.879082] LustreError: 358496:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12238.885876] LustreError: 358496:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12238.949213] LustreError: 371687:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12238.969496] LustreError: 371687:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.143@tcp: cfg command failed: rc = -114 [12238.982318] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12238.982318] [12560.352343] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12560.355524] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12560.371951] Lustre: Skipped 2 previous similar messages [12560.376881] Lustre: Skipped 1 previous similar message [12566.276721] Lustre: server umount lustre-MDT0000 complete [12570.015727] LustreError: 358486:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789141618 with bad export cookie 12314018528253270084 [12570.021850] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12570.028914] LustreError: 358486:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12570.593637] LustreError: 364450:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12570.594413] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12570.631524] LustreError: 364450:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 7 previous similar messages [12570.672824] Lustre: Skipped 1 previous similar message [12570.682818] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12570.689456] Lustre: Skipped 5 previous similar messages [12576.615580] Lustre: server umount lustre-MDT0001 complete [12581.296135] Lustre: server umount lustre-OST0000 complete [12591.992783] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_hostid [12604.356575] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [12644.537515] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 11:48:10 (1789141690) [12651.929823] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12652.190499] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12652.220446] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12652.297510] Lustre: lustre-MDT0000: new disk, initializing [12652.390080] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12652.425709] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12657.334863] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [12668.443340] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12668.644062] Lustre: 382435:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12668.776779] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12668.786847] Lustre: Skipped 1 previous similar message [12668.882443] Lustre: lustre-MDT0001: new disk, initializing [12669.075736] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12669.126928] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12669.154816] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12674.048450] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [12684.819968] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12691.005671] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12699.035108] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12699.325823] Lustre: lustre-OST0000: new disk, initializing [12699.329248] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12699.334968] Lustre: 384403:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12699.445807] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12704.834984] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12704.846017] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12704.983944] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12705.999377] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_default_debug -1 all [12716.992925] Lustre: DEBUG MARKER: oleg643-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12730.336474] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12730.342293] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12730.364032] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12730.371625] Lustre: Skipped 1 previous similar message [12732.196423] Lustre: server umount lustre-MDT0000 complete [12735.973086] LustreError: 382446:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12736.019585] LustreError: 382446:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [12736.159829] LustreError: 382426:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789141785 with bad export cookie 12314018528253272037 [12736.175702] LustreError: MGC192.168.206.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12736.180311] LustreError: 382426:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12736.714373] Lustre: server umount lustre-MDT0001 complete [12750.541930] Lustre: server umount lustre-OST0000 complete [12783.393898] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing set_hostid [12795.143621] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing load_modules_local [12825.108995] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12562 sec ========= 11:51:10 (1789141870) [12827.207611] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 11:51:13 (1789141873) === [12831.480860] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 11:51:17 (1789141877) === [12873.796267] Lustre: DEBUG MARKER: oleg643-server.virtnet: executing unload_modules_local [12876.417795] Key type lgssc unregistered [12876.712729] LNet: 390775:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12876.722769] LNetError: 390775:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12876.749118] LNet: Removed LNI 192.168.206.143@tcp [12877.653205] Key type .llcrypt unregistered [12877.655455] Key type ._llcrypt unregistered