[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 478195531 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2560MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.003175] x2apic enabled [ 0.004008] Switched APIC routing to physical x2apic. [ 0.005011] kvm-guest: setup PV IPIs [ 0.008248] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009015] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010007] pid_max: default: 32768 minimum: 301 [ 0.011126] LSM: Security Framework initializing [ 0.012027] Yama: becoming mindful. [ 0.013020] SELinux: Initializing. [ 0.013631] *** VALIDATE selinux *** [ 0.020569] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.025184] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.026128] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027121] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028115] *** VALIDATE tmpfs *** [ 0.029441] *** VALIDATE proc *** [ 0.031254] *** VALIDATE cgroup *** [ 0.032008] *** VALIDATE cgroup2 *** [ 0.034215] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035150] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036005] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037021] Spectre V2 : User space: Vulnerable [ 0.038004] Speculative Store Bypass: Vulnerable [ 0.041287] debug: unmapping init [mem 0xffffffff9a459000-0xffffffff9a460fff] [ 0.044161] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045802] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046020] ... version: 2 [ 0.047011] ... bit width: 48 [ 0.047958] ... generic registers: 4 [ 0.048016] ... value mask: 0000ffffffffffff [ 0.049009] ... max period: 00007fffffffffff [ 0.050013] ... fixed-purpose events: 3 [ 0.051012] ... event mask: 000000070000000f [ 0.052310] rcu: Hierarchical SRCU implementation. [ 0.054260] smp: Bringing up secondary CPUs ... [ 0.055458] x86: Booting SMP configuration: [ 0.056023] .... node #0, CPUs: #1 #2 #3 [ 0.062072] smp: Brought up 1 node, 4 CPUs [ 0.064013] smpboot: Max logical packages: 1 [ 0.065011] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.124334] node 0 deferred pages initialised in 57ms [ 0.128260] devtmpfs: initialized [ 0.130347] x86/mm: Memory block size: 128MB [ 0.134414] gcov: version magic: 0x41383552 [ 0.137312] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.141187] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.144415] pinctrl core: initialized pinctrl subsystem [ 0.147285] [ 0.147741] ************************************************************* [ 0.149009] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.151010] ** ** [ 0.154070] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.156008] ** ** [ 0.158009] ** This means that this kernel is built to expose internal ** [ 0.160013] ** IOMMU data structures, which may compromise security on ** [ 0.163015] ** your system. ** [ 0.166010] ** ** [ 0.169014] ** If you see this message and you are not debugging the ** [ 0.172013] ** kernel, report this immediately to your vendor! ** [ 0.175012] ** ** [ 0.178011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.181013] ************************************************************* [ 0.184819] NET: Registered protocol family 16 [ 0.187561] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.190129] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.194089] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.199081] cpuidle: using governor menu [ 0.200835] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.201553] PCI: Using configuration type 1 for base access [ 0.203189] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.213107] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.215012] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.218156] cryptd: max_cpu_qlen set to 1000 [ 0.221205] ACPI: Added _OSI(Module Device) [ 0.223016] ACPI: Added _OSI(Processor Device) [ 0.224010] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.226015] ACPI: Added _OSI(Processor Aggregator Device) [ 0.231659] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.238442] ACPI: Interpreter enabled [ 0.240153] ACPI: PM: (supports S0 S3 S4 S5) [ 0.242013] ACPI: Using IOAPIC for interrupt routing [ 0.244097] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.248477] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.259327] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.261030] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.263014] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.268080] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.273256] acpiphp: Slot [2] registered [ 0.274068] acpiphp: Slot [5] registered [ 0.275070] acpiphp: Slot [6] registered [ 0.276057] acpiphp: Slot [7] registered [ 0.277091] acpiphp: Slot [8] registered [ 0.279095] acpiphp: Slot [9] registered [ 0.280092] acpiphp: Slot [10] registered [ 0.282100] acpiphp: Slot [3] registered [ 0.283096] acpiphp: Slot [4] registered [ 0.285090] acpiphp: Slot [11] registered [ 0.286123] acpiphp: Slot [12] registered [ 0.288104] acpiphp: Slot [13] registered [ 0.289133] acpiphp: Slot [14] registered [ 0.290056] acpiphp: Slot [15] registered [ 0.292083] acpiphp: Slot [16] registered [ 0.293081] acpiphp: Slot [17] registered [ 0.294093] acpiphp: Slot [18] registered [ 0.296100] acpiphp: Slot [19] registered [ 0.297096] acpiphp: Slot [20] registered [ 0.299158] acpiphp: Slot [21] registered [ 0.301102] acpiphp: Slot [22] registered [ 0.302101] acpiphp: Slot [23] registered [ 0.304106] acpiphp: Slot [24] registered [ 0.306134] acpiphp: Slot [25] registered [ 0.308135] acpiphp: Slot [26] registered [ 0.309116] acpiphp: Slot [27] registered [ 0.311114] acpiphp: Slot [28] registered [ 0.312094] acpiphp: Slot [29] registered [ 0.314136] acpiphp: Slot [30] registered [ 0.316118] acpiphp: Slot [31] registered [ 0.317091] PCI host bridge to bus 0000:00 [ 0.319018] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.321023] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.324024] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.327026] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.330023] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.333028] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.335269] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.339093] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.342533] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.353020] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.361014] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.364018] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.366027] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.369016] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.372532] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.375881] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.378039] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.380838] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.386014] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.403015] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.409014] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.416000] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.424015] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.431278] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.447016] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.458013] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.470018] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.478020] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.496016] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.509466] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.515017] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.521014] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.541021] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.559244] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.567078] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.581016] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.614016] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.633957] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.650019] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.672017] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.717018] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.737804] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.751016] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.763016] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.781017] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.795136] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.797819] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.800339] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.802371] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.805198] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.809091] iommu: Default domain type: Passthrough [ 0.812456] SCSI subsystem initialized [ 0.814133] ACPI: bus type USB registered [ 0.817123] usbcore: registered new interface driver usbfs [ 0.820090] usbcore: registered new interface driver hub [ 0.823082] usbcore: registered new device driver usb [ 0.825136] pps_core: LinuxPPS API ver. 1 registered [ 0.827009] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.829067] PTP clock support registered [ 0.831135] EDAC MC: Ver: 3.0.0 [ 0.832311] PCI: Using ACPI for IRQ routing [ 0.833712] NetLabel: Initializing [ 0.834010] NetLabel: domain hash size = 128 [ 0.835008] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.836059] NetLabel: unlabeled traffic allowed by default [ 0.837189] vgaarb: loaded [ 0.838268] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.839015] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.844684] clocksource: Switched to clocksource kvm-clock [ 0.980540] VFS: Disk quotas dquot_6.6.0 [ 0.982071] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.983986] *** VALIDATE ramfs *** [ 0.985360] *** VALIDATE hugetlbfs *** [ 0.986727] pnp: PnP ACPI init [ 0.988794] pnp: PnP ACPI: found 6 devices [ 1.004039] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.006871] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.008813] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.010719] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.013297] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.015374] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.017765] NET: Registered protocol family 2 [ 1.020914] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.027197] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.031928] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.038246] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.042532] TCP: Hash tables configured (established 65536 bind 65536) [ 1.045753] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.049168] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.051498] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.054253] NET: Registered protocol family 1 [ 1.057703] RPC: Registered named UNIX socket transport module. [ 1.059524] RPC: Registered udp transport module. [ 1.060974] RPC: Registered tcp transport module. [ 1.062636] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.065493] NET: Registered protocol family 44 [ 1.066947] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.069220] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.071048] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.072947] PCI: CLS 0 bytes, default 64 [ 1.074485] Unpacking initramfs... [ 2.499537] debug: unmapping init [mem 0xffff922e3cc54000-0xffff922e3ffbffff] [ 2.503745] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.505980] software IO TLB: mapped [mem 0x00000000b8c54000-0x00000000bcc54000] (64MB) [ 2.509044] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.025695] Initialise system trusted keyrings [ 3.028645] Key type blacklist registered [ 3.030207] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.040396] zbud: loaded [ 3.043177] *** VALIDATE nfs *** [ 3.044182] *** VALIDATE nfs4 *** [ 3.045176] pstore: using deflate compression [ 3.048159] Platform Keyring initialized [ 3.141973] NET: Registered protocol family 38 [ 3.145302] Key type asymmetric registered [ 3.147174] Asymmetric key parser 'x509' registered [ 3.148959] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.152533] io scheduler mq-deadline registered [ 3.154460] io scheduler kyber registered [ 3.156240] io scheduler bfq registered [ 3.158122] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.161129] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.163558] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.166167] ACPI: Power Button [PWRF] [ 3.171559] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.178722] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.191465] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.197887] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.212234] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.242403] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.273358] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.278526] Non-volatile memory driver v1.3 [ 3.280367] Linux agpgart interface v0.103 [ 3.312747] virtio_blk virtio1: [vda] 146704 512-byte logical blocks (75.1 MB/71.6 MiB) [ 3.316171] vda: detected capacity change from 0 to 75112448 [ 3.332822] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.336690] vdb: detected capacity change from 0 to 1073741824 [ 3.360222] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.364395] vdc: detected capacity change from 0 to 2621440000 [ 3.387924] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.391607] vdd: detected capacity change from 0 to 2621440000 [ 3.408407] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.412154] vde: detected capacity change from 0 to 4294967296 [ 3.429813] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.433327] vdf: detected capacity change from 0 to 4294967296 [ 3.445235] libphy: Fixed MDIO Bus: probed [ 3.494600] usbcore: registered new interface driver usbserial_generic [ 3.500582] usbserial: USB Serial support registered for generic [ 3.507942] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.514532] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.517502] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.522183] mousedev: PS/2 mouse device common for all mice [ 3.526294] rtc_cmos 00:05: RTC can wake from S4 [ 3.529992] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.530828] rtc_cmos 00:05: registered as rtc0 [ 3.540145] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.541097] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.548929] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.552688] intel_pstate: CPU model not supported [ 3.559650] hid: raw HID events driver (C) Jiri Kosina [ 3.562466] usbcore: registered new interface driver usbhid [ 3.565139] usbhid: USB HID core driver [ 3.567236] drop_monitor: Initializing network drop monitor service [ 3.570279] Initializing XFRM netlink socket [ 3.572844] NET: Registered protocol family 10 [ 3.576353] Segment Routing with IPv6 [ 3.578336] NET: Registered protocol family 17 [ 3.580683] mpls_gso: MPLS GSO support [ 3.588790] RAS: Correctable Errors collector initialized. [ 3.590157] AVX version of gcm_enc/dec engaged. [ 3.591089] AES CTR mode by8 optimization enabled [ 3.687761] sched_clock: Marking stable (3687673479, 0)->(4435938247, -748264768) [ 3.693300] registered taskstats version 1 [ 3.695590] Loading compiled-in X.509 certificates [ 3.698948] zswap: loaded using pool lzo/zbud [ 3.742107] Key type big_key registered [ 3.757581] Key type encrypted registered [ 3.759504] ima: No TPM chip found, activating TPM-bypass! [ 3.762120] ima: Allocated hash algorithm: sha1 [ 3.764145] ima: No architecture policies found [ 3.766414] evm: Initialising EVM extended attributes: [ 3.768407] evm: security.selinux [ 3.769709] evm: security.ima [ 3.770852] evm: security.capability [ 3.772277] evm: HMAC attrs: 0x1 [ 3.776498] rtc_cmos 00:05: setting system clock to 2026-09-18 23:53:35 UTC (1789775615) [ 3.783206] debug: unmapping init [mem 0xffffffff9b403000-0xffffffff9b5fffff] [ 3.786350] debug: unmapping init [mem 0xffffffff9a182000-0xffffffff9a458fff] [ 3.795072] Write protecting the kernel read-only data: 28672k [ 3.798619] debug: unmapping init [mem 0xffffffff98803000-0xffffffff989fffff] [ 3.801547] debug: unmapping init [mem 0xffffffff99114000-0xffffffff991fffff] [ 3.846479] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.856953] systemd[1]: Detected virtualization kvm. [ 3.858781] systemd[1]: Detected architecture x86-64. [ 3.860954] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.890616] systemd[1]: No hostname configured. [ 3.892715] systemd[1]: Set hostname to . [ 3.895601] random: systemd: uninitialized urandom read (16 bytes read) [ 3.898807] systemd[1]: Initializing machine ID from random generator. [ 3.986801] random: ln: uninitialized urandom read (6 bytes read) [ 4.116723] random: systemd: uninitialized urandom read (16 bytes read) [ 4.120211] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 4.126746] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 4.133107] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Slices. [ OK ] Reached target Swap. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket. Starting Journal Service... [ OK ] Reached target Sockets. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... Starting Setup Virtual Console... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.143632] device-mapper: uevent: version 1.0.3 [ 5.145840] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ 5.869598] virtio_net virtio0 ens2: renamed from eth0 [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 6.004070] random: fast init done [ 6.106328] scsi host0: ata_piix [ 6.132729] scsi host1: ata_piix [ 6.134439] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 6.137323] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.796463] dracut-initqueue[593]: RTNETLINK answers: File exists [ 11.281454] random: crng init done [ 11.282658] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 11.846218] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 13.081580] printk: systemd: 25 output lines suppressed due to ratelimiting [ 13.307599] SELinux: Disabled at runtime. [ 13.365834] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 13.375475] systemd[1]: Detected virtualization kvm. [ 13.377298] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 13.902849] systemd[1]: initrd-switch-root.service: Succeeded. [ 13.907433] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 13.912854] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 13.917786] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 13.923195] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 13.934595] systemd[1]: Starting Journal Service... Starting Journal Service... [ 13.940300] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice User and Session Slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on Process Core Dump Socket. Starting Create list of required st…ce nodes for the current kernel... Mounting Kernel Debug File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Created slice system-getty.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Stopped target Switch Root. Starting Remount Root and Kernel File Systems... Mounting Huge Pages File System... [ OK ] Listening on initctl Compatibility Named Pipe. Activating swap /dev/disk/by-label/SWAP... [ OK ] Stopped target Initrd Root File System. [ OK ] Reached target Local Encrypted Volumes. Mounting POSIX Message Queue File System... [ 14.137349] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... Starting Apply Kernel Variables... [ OK ] Stopped target Initrd File Systems. [ OK ] Reached target Slices. [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 14.554819] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 14.944784] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 14.980565] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 15.568580] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 15.593289] EDAC sbridge: Ver: 1.1.2 [ 17.141870] Key type dns_resolver registered [ 17.520579] NFS: Registering the id_resolver key type [ 17.522580] Key type id_resolver registered [ 17.523799] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Load/Save Random Seed. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started irqbalance daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting System Logging Service... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg211-server login: [ 59.648164] hrtimer: interrupt took 2396766 ns [ 76.655828] libcfs: loading out-of-tree module taints kernel. [ 76.748929] Key type ._llcrypt registered [ 76.753954] Key type .llcrypt registered [ 76.880975] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 93.934103] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 95.571955] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 95.604654] alg: No test for adler32 (adler32-zlib) [ 97.197965] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [ 98.375501] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 100.127246] Key type lgssc registered [ 102.461361] Lustre: Echo OBD driver; http://www.lustre.org/ [ 121.382882] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 163.306737] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 176.732388] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 176.776256] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 178.160890] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 178.218090] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 178.363668] Lustre: lustre-MDT0000: new disk, initializing [ 178.505418] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 178.533830] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 183.194443] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 197.812050] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 197.932160] Lustre: 6518:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 197.956876] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 197.960580] Lustre: Skipped 1 previous similar message [ 198.041241] Lustre: lustre-MDT0001: new disk, initializing [ 198.100164] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 198.146813] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 198.164403] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 203.019443] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 207.831907] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 218.046053] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 218.397778] Lustre: lustre-OST0000: new disk, initializing [ 218.402683] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 218.409804] Lustre: 8458:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 218.486369] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 225.805958] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 226.934236] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 226.944569] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 227.036386] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 239.918150] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 240.106279] Lustre: lustre-OST0001: new disk, initializing [ 240.109961] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 240.116464] Lustre: 9532:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 240.203950] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 246.807594] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 246.814375] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 246.849470] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 247.770250] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 260.938990] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 270.000240] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 276.485783] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing check_logdir /tmp/testlogs/ [ 281.246586] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing yml_node [ 285.277258] Lustre: DEBUG MARKER: Client: 2.17.57.110 [ 287.719979] Lustre: DEBUG MARKER: MDS: 2.17.57.110 [ 290.594687] Lustre: DEBUG MARKER: OSS: 2.17.57.110 [ 291.969817] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri Sep 18 19:58:21 EDT 2026 [ 310.827867] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 312.809477] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 328.672614] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 328.674583] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 328.687506] Lustre: Skipped 3 previous similar messages [ 328.698884] Lustre: Skipped 3 previous similar messages [ 333.794246] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 333.807040] Lustre: Skipped 3 previous similar messages [ 334.749965] Lustre: server umount lustre-MDT0000 complete [ 338.617649] LustreError: 6512:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789775950 with bad export cookie 13067296187165210237 [ 338.625472] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 338.629541] LustreError: 6512:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 338.915436] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 338.930573] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 338.935323] Lustre: Skipped 1 previous similar message [ 344.034100] LustreError: 8464:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 344.038081] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 344.064874] LustreError: 8464:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 344.096393] Lustre: Skipped 1 previous similar message [ 345.202513] Lustre: server umount lustre-MDT0001 complete [ 350.025655] Lustre: server umount lustre-OST0000 complete [ 354.015244] Lustre: server umount lustre-OST0001 complete [ 361.382151] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 371.972871] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 411.490540] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 411.740280] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 411.768746] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 411.890314] Lustre: lustre-MDT0000: new disk, initializing [ 411.965704] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 411.977778] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 416.491074] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 427.408646] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 427.535914] Lustre: 15086:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 427.597861] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 427.601597] Lustre: Skipped 1 previous similar message [ 427.671050] Lustre: lustre-MDT0001: new disk, initializing [ 427.771885] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 427.821320] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 427.833959] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 432.529480] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 445.209618] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 452.041139] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 459.485090] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 459.762284] Lustre: lustre-OST0000: new disk, initializing [ 459.765498] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 459.779175] Lustre: 17069:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 459.887192] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 461.575324] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 461.589552] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 461.718053] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 465.894312] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 476.156732] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 481.398464] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 481.732845] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 486.658524] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 486.844657] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 492.513878] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 492.521028] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 492.525358] LustreError: Skipped 1 previous similar message [ 492.530928] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 492.576925] Lustre: Skipped 2 previous similar messages [ 494.871660] Lustre: server umount lustre-OST0000 complete [ 499.680792] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 499.691553] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 502.755853] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 502.779477] Lustre: Skipped 2 previous similar messages [ 505.299862] Lustre: server umount lustre-MDT0000 complete [ 507.880199] LustreError: 15096:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 507.929206] LustreError: 15096:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 509.594108] LustreError: 15080:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789776121 with bad export cookie 13067296187165211770 [ 509.599133] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 509.609349] LustreError: 15080:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 510.143221] Lustre: server umount lustre-MDT0001 complete [ 515.696594] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 20:02:05 (1789776125) [ 522.927635] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 523.563590] LustreError: 19337:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 523.646261] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 528.813473] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 528.866887] LustreError: 19338:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 533.994658] LustreError: 19337:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 537.461567] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 542.163932] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 550.722673] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 556.453609] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 563.505680] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 570.304393] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 579.896025] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 587.057259] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 594.404155] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 594.415585] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 594.434647] Lustre: Skipped 1 previous similar message [ 594.446810] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 599.546738] Lustre: server umount lustre-MDT0000 complete [ 602.841189] LustreError: 19318:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789776214 with bad export cookie 13067296187165212652 [ 602.856174] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 602.860207] LustreError: 19318:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 603.156683] Lustre: server umount lustre-MDT0001 complete [ 616.936288] Lustre: server umount lustre-OST0000 complete [ 637.960734] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 651.602399] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 651.908713] LustreError: 23718:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 652.019170] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 652.026358] Lustre: Skipped 2 previous similar messages [ 657.904520] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 668.617341] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 674.805709] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 678.430312] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 686.361555] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 693.606103] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 702.478909] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 702.769509] Lustre: lustre-OST0001: new disk, initializing [ 702.775345] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 702.782424] Lustre: 26035:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 704.653561] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 704.672193] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 704.747894] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 709.874809] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 720.405531] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 724.640544] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 740.839918] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 740.845976] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 740.850214] Lustre: Skipped 4 previous similar messages [ 740.851054] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 740.853235] Lustre: Skipped 6 previous similar messages [ 744.093907] Lustre: server umount lustre-MDT0000 complete [ 745.959559] LustreError: 24450:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 745.972358] LustreError: 24450:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 747.751545] LustreError: 23697:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789776359 with bad export cookie 13067296187165213499 [ 747.761256] LustreError: 23697:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 747.767041] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 748.163210] Lustre: server umount lustre-MDT0001 complete [ 761.586058] Lustre: server umount lustre-OST0000 complete [ 775.385514] Lustre: server umount lustre-OST0001 complete [ 793.069877] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 803.970288] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 804.293108] LustreError: 29055:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 804.413320] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 804.418671] Lustre: Skipped 3 previous similar messages [ 809.363385] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 819.175811] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 823.906484] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 826.799230] Lustre: 30204:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 833.385078] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 840.152404] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 848.104394] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 854.813805] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 863.162839] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 867.100444] Lustre: 32082:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 874.988829] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 875.006847] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 875.009054] Lustre: Skipped 2 previous similar messages [ 875.035519] Lustre: Skipped 3 previous similar messages [ 879.315473] Lustre: server umount lustre-MDT0000 complete [ 880.113407] LustreError: 32285:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 880.150135] LustreError: 32285:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 884.083988] LustreError: 29036:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789776495 with bad export cookie 13067296187165214794 [ 884.104833] LustreError: 29036:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 884.116494] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 884.609743] Lustre: server umount lustre-MDT0001 complete [ 899.209946] Lustre: server umount lustre-OST0000 complete [ 900.385991] Lustre: 3651:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789776496/real 1789776496] req@ffff922d820bd880 x1876715855184512/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789776512 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 903.357245] Lustre: server umount lustre-OST0001 complete [ 911.800710] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 20:08:41 (1789776521) [ 944.218941] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 954.388838] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 959.647348] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 969.310689] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 974.700085] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 977.915076] Lustre: 36036:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 985.282102] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 992.604846] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1003.376699] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1011.057359] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1018.917669] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1022.556983] Lustre: 37923:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1029.612300] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1029.616634] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1029.629696] Lustre: Skipped 5 previous similar messages [ 1029.648783] Lustre: Skipped 3 previous similar messages [ 1034.731099] LustreError: 34883:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1034.760615] LustreError: 34883:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 1034.874431] Lustre: server umount lustre-MDT0000 complete [ 1038.700065] LustreError: 34867:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789776650 with bad export cookie 13067296187165215900 [ 1038.719772] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1038.726577] LustreError: 34867:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1039.263700] Lustre: server umount lustre-MDT0001 complete [ 1052.421345] Lustre: server umount lustre-OST0000 complete [ 1055.713108] Lustre: 3652:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789776651/real 1789776651] req@ffff922e9f6c2300 x1876715855233920/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789776667 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1056.394991] Lustre: server umount lustre-OST0001 complete [ 1065.381473] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 20:11:14 (1789776674) [ 1083.801496] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1093.160318] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1093.879435] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1093.895306] Lustre: Skipped 7 previous similar messages [ 1099.064304] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1109.523455] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1115.221369] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1118.410127] Lustre: 41215:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1127.003608] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1134.637062] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1144.520314] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1152.263580] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1161.136122] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1165.747756] Lustre: 43092:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1166.768940] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1166.775870] Lustre: Skipped 1 previous similar message [ 1170.401680] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1170.408480] Lustre: Skipped 2 previous similar messages [ 1176.525532] Lustre: server umount lustre-MDT0000 complete [ 1181.284637] LustreError: 40046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789776793 with bad export cookie 13067296187165217006 [ 1181.308377] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1181.309285] LustreError: 40046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1187.836200] Lustre: server umount lustre-MDT0001 complete [ 1195.944789] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1212.243335] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1216.821159] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1222.224145] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1227.013888] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1227.772551] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1227.779238] Lustre: Skipped 1 previous similar message [ 1235.975227] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1237.269613] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1244.205291] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1246.523607] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1253.377491] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1258.660192] Lustre: server umount lustre-MDT0000 complete [ 1262.559731] LustreError: 40046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789776874 with bad export cookie 13067296187165218133 [ 1262.571741] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1262.575549] LustreError: 40046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 1262.854948] Lustre: server umount lustre-MDT0001 complete [ 1276.499468] Lustre: server umount lustre-OST0000 complete [ 1279.969452] Lustre: 3651:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789776875/real 1789776875] req@ffff922d8237a680 x1876715855324288/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789776891 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1280.439683] Lustre: server umount lustre-OST0001 complete [ 1290.564796] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 20:14:59 (1789776899) [ 1307.787255] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1321.132601] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1321.617566] LustreError: 47703:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1321.655545] LustreError: 47703:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 34 previous similar messages [ 1327.219426] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1337.417808] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1344.217242] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1348.700713] Lustre: 48857:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1356.187756] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1363.909480] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1373.188874] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1381.460626] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1391.292432] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1396.795628] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1412.805845] Lustre: server umount lustre-MDT0000 complete [ 1414.625596] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1414.626703] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1414.657571] Lustre: Skipped 12 previous similar messages [ 1416.984184] LustreError: 47685:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789777028 with bad export cookie 13067296187165218931 [ 1416.995327] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1417.007865] LustreError: 47685:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1417.394582] Lustre: server umount lustre-MDT0001 complete [ 1432.251620] Lustre: server umount lustre-OST0000 complete [ 1436.134929] Lustre: 3653:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789777031/real 1789777031] req@ffff922e836cf800 x1876715855384960/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789777047 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1436.540519] Lustre: server umount lustre-OST0001 complete [ 1448.254340] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 20:17:37 (1789777057) [ 1451.239660] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1453.284461] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 20:17:42 (1789777062) [ 1464.569485] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 1475.052928] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1499.143618] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1499.474745] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1499.541120] Lustre: lustre-MDT0000: new disk, initializing [ 1499.659768] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1504.650608] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1515.683933] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1523.053892] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1523.348760] Lustre: lustre-OST0000: new disk, initializing [ 1523.352177] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1523.359876] Lustre: Skipped 1 previous similar message [ 1523.364972] Lustre: 54935:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1524.907460] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1524.916725] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1524.978182] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1530.500597] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1540.052808] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1542.712718] Lustre: DEBUG MARKER: create test files [ 1581.539691] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1581.559668] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1581.567779] Lustre: Skipped 19 previous similar messages [ 1583.055563] Lustre: server umount lustre-OST0000 complete [ 1588.108645] Lustre: server umount lustre-MDT0000 complete [ 1601.454588] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 1604.991902] Key type lgssc unregistered [ 1605.464045] LNet: 56565:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1605.484082] LNetError: 56565:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1605.503876] LNet: Removed LNI 192.168.202.111@tcp [ 1606.602170] Key type .llcrypt unregistered [ 1606.603828] Key type ._llcrypt unregistered [ 1637.858373] Key type ._llcrypt registered [ 1637.862059] Key type .llcrypt registered [ 1638.062618] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1639.351451] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1639.388516] alg: No test for adler32 (adler32-zlib) [ 1640.433427] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [ 1640.646937] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 1642.304556] Key type lgssc registered [ 1643.220862] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1650.530698] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1650.574536] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1652.144944] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1656.073397] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1663.195945] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1669.396665] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1669.912476] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1676.108834] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1677.110389] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:101 to 0x240000400:129) [ 1677.263913] Lustre: 59298:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1677.289149] Lustre: 59298:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1677.297913] Lustre: 59298:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1677.304806] Lustre: 59298:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1677.318673] Lustre: 59298:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1677.334712] Lustre: 59298:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1683.964852] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1689.880932] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1695.783843] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1728.490037] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1728.494881] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1728.501881] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1733.605133] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1734.557970] Lustre: server umount lustre-OST0000 complete [ 1738.996960] Lustre: server umount lustre-MDT0000 complete [ 1752.549645] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 1756.115502] Key type lgssc unregistered [ 1756.366252] LNet: 60925:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1756.373580] LNetError: 60925:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1757.417562] LNet: Removed LNI 192.168.202.111@tcp [ 1758.557265] Key type .llcrypt unregistered [ 1758.561037] Key type ._llcrypt unregistered [ 1792.242763] Key type ._llcrypt registered [ 1792.245616] Key type .llcrypt registered [ 1792.340612] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1793.760593] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1793.786171] alg: No test for adler32 (adler32-zlib) [ 1794.936468] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [ 1795.288133] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 1796.991399] Key type lgssc registered [ 1798.652898] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1806.835869] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1806.856514] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1808.364146] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1814.060376] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1823.539861] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1832.489173] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1833.037603] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1834.101766] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1834.231713] Lustre: 63757:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1834.252981] Lustre: 63757:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1834.268775] Lustre: 63757:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1834.285235] Lustre: 63757:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1834.297805] Lustre: 63757:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1834.330240] Lustre: 63757:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1840.606363] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1848.886729] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1855.384833] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1870.306301] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1870.313579] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1870.326857] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1871.971026] Lustre: server umount lustre-OST0000 complete [ 1876.941309] Lustre: server umount lustre-MDT0000 complete [ 1888.489844] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 1891.850807] Key type lgssc unregistered [ 1892.264114] LNet: 65313:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1892.274847] LNetError: 65313:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1892.292769] LNet: Removed LNI 192.168.202.111@tcp [ 1893.400394] Key type .llcrypt unregistered [ 1893.407665] Key type ._llcrypt unregistered [ 1916.697870] Key type ._llcrypt registered [ 1916.701347] Key type .llcrypt registered [ 1916.902266] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 1935.744402] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1937.429035] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1937.516205] alg: No test for adler32 (adler32-zlib) [ 1938.686922] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [ 1938.918387] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 1940.631313] Key type lgssc registered [ 1941.828281] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1984.883322] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1984.909861] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1986.248328] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1986.302177] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1986.417936] Lustre: lustre-MDT0000: new disk, initializing [ 1986.585424] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1986.751716] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1993.234058] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2007.316590] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2007.504939] Lustre: 68468:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2007.616706] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2007.621058] Lustre: Skipped 1 previous similar message [ 2007.740912] Lustre: lustre-MDT0001: new disk, initializing [ 2007.933862] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2007.987288] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2007.996535] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2013.707195] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2023.598078] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2030.565104] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2037.402235] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2037.708611] Lustre: lustre-OST0000: new disk, initializing [ 2037.715223] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2037.728695] Lustre: 70438:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2037.845164] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2045.002322] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2045.013385] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2045.014902] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2045.173804] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2055.330074] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2060.455069] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2060.691611] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2065.818681] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2066.102231] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2071.022310] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2071.029898] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2071.031837] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2071.041416] Lustre: Skipped 1 previous similar message [ 2071.060475] LustreError: Skipped 1 previous similar message [ 2074.467069] Lustre: server umount lustre-OST0000 complete [ 2079.719304] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2079.729702] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2079.752182] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2079.764947] Lustre: Skipped 1 previous similar message [ 2081.257125] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2081.259573] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2081.301967] Lustre: Skipped 1 previous similar message [ 2084.886743] Lustre: server umount lustre-MDT0000 complete [ 2088.960201] LustreError: 68460:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789777700 with bad export cookie 4776978526920403392 [ 2088.979817] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2089.217687] Lustre: server umount lustre-MDT0001 complete [ 2099.772347] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 20:28:29 (1789777709) [ 2112.913794] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 2156.378343] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 2168.532765] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 2206.387290] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2206.555372] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2206.576126] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2206.638906] Lustre: lustre-MDT0000: new disk, initializing [ 2206.708495] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2206.734706] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2211.132632] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2222.301860] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2222.451790] Lustre: 76462:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2222.530669] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2222.538680] Lustre: Skipped 1 previous similar message [ 2222.622827] Lustre: lustre-MDT0001: new disk, initializing [ 2222.715948] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2222.765453] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2222.782466] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2227.820830] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2238.288814] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2244.928769] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2252.606380] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2252.984086] Lustre: lustre-OST0000: new disk, initializing [ 2252.992766] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2253.015541] Lustre: 78434:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2253.140897] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2260.301451] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2260.597226] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2260.607647] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2260.749913] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2269.499639] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2275.475150] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2275.832774] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2281.169374] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2281.339366] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2286.564914] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2286.568760] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2286.581649] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2286.590529] Lustre: Skipped 1 previous similar message [ 2286.606966] Lustre: Skipped 1 previous similar message [ 2289.624485] Lustre: server umount lustre-OST0000 complete [ 2294.240927] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2294.256930] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2294.278116] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2300.272742] Lustre: server umount lustre-MDT0000 complete [ 2301.926180] LustreError: 76474:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2301.942223] LustreError: 76474:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2304.842125] LustreError: 77258:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789777916 with bad export cookie 4776978526920404267 [ 2304.859169] LustreError: 77258:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2304.876915] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2305.421135] Lustre: server umount lustre-MDT0001 complete [ 2317.513365] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 20:32:06 (1789777926) [ 2325.579139] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2326.107905] LustreError: 80850:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2326.207857] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2331.618551] LustreError: 80851:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2331.854646] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2335.731703] LustreError: 80850:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2340.832955] LustreError: 80851:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2341.297867] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2347.129940] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2357.584538] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2366.712501] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2376.804775] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2377.349617] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2377.359970] Lustre: Skipped 1 previous similar message [ 2383.922646] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2392.446702] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2397.138304] Lustre: 83525:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting f71bf6db-1e6f-4b5b-b31d-7f552266ad3f at adminstrative request [ 2401.250621] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2401.271953] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2410.472985] Lustre: MGS: Client f71bf6db-1e6f-4b5b-b31d-7f552266ad3f (at 0@lo) reconnecting [ 2410.493212] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x424b3ca924c270a4 to 0x424b3ca924c27344 [ 2410.529262] Lustre: MGC192.168.202.111@tcp: Connection restored to 0@lo (at 0@lo) [ 2434.928356] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2435.169544] Lustre: lustre-OST0001: new disk, initializing [ 2435.172417] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2435.177522] Lustre: 83920:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2440.763846] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2440.772833] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2440.940327] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2441.193169] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2450.632795] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2456.547101] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2456.553662] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2456.560065] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2456.560872] Lustre: Skipped 2 previous similar messages [ 2456.570105] Lustre: Skipped 3 previous similar messages [ 2459.776643] Lustre: server umount lustre-OST0001 complete [ 2461.681482] LustreError: 82672:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2461.701845] LustreError: 82672:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2469.831957] Lustre: server umount lustre-OST0000 complete [ 2474.976352] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2474.985455] LustreError: Skipped 2 previous similar messages [ 2474.992942] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2475.019100] Lustre: Skipped 2 previous similar messages [ 2475.028171] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2475.039192] Lustre: Skipped 3 previous similar messages [ 2479.790170] Lustre: server umount lustre-MDT0000 complete [ 2482.155323] LustreError: 80851:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2482.190824] LustreError: 80851:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2484.179096] LustreError: 80831:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789778095 with bad export cookie 4776978526920405828 [ 2484.188189] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2484.194363] LustreError: 80831:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2484.794526] Lustre: server umount lustre-MDT0001 complete [ 2496.444930] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 20:35:05 (1789778105) [ 2497.713613] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2499.767088] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 20:35:09 (1789778109) [ 2501.598515] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2503.667346] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 20:35:13 (1789778113) [ 2505.732321] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2507.771617] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 20:35:17 (1789778117) [ 2509.537498] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2511.554410] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 20:35:20 (1789778120) [ 2513.376501] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2519.216232] loop: module loaded [ 2519.233643] loop0: detected capacity change from 0 to 4294967296 [ 2519.285175] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2520.959348] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2524.286095] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2527.782626] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2555.446387] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2555.503111] LustreError: 88042:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2555.514627] LustreError: 88042:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2555.530781] LustreError: 88042:0:(tgt_mount.c:2582:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2555.546644] LustreError: 88042:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2575.782397] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 2586.188346] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 2626.776489] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2627.117460] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2627.148241] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2627.273338] Lustre: lustre-MDT0000: new disk, initializing [ 2627.449305] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2627.462763] Lustre: Skipped 1 previous similar message [ 2627.505066] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2633.125495] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2645.710454] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2645.819637] Lustre: 90912:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2646.118199] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2651.751280] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2663.405878] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2670.461235] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2679.149108] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2679.397876] Lustre: 92880:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2686.911412] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2688.078399] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2688.278430] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2697.204726] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2703.268729] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2703.634590] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2709.680886] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2709.914057] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2714.084710] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2714.087743] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2714.102805] LustreError: Skipped 1 previous similar message [ 2714.106379] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2714.106388] Lustre: Skipped 2 previous similar messages [ 2714.169802] Lustre: Skipped 2 previous similar messages [ 2726.367618] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2726.829602] Lustre: server umount lustre-OST0000 complete [ 2737.499349] Lustre: server umount lustre-MDT0000 complete [ 2739.685783] LustreError: 90923:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2739.726504] LustreError: 90923:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2741.685201] LustreError: 91708:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789778353 with bad export cookie 4776978526920406444 [ 2741.686527] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2741.695141] LustreError: 91708:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2742.044607] Lustre: server umount lustre-MDT0001 complete [ 2751.718138] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 20:39:21 (1789778361) [ 2759.030696] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2759.600492] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 2759.609806] Lustre: Skipped 2 previous similar messages [ 2764.005300] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2773.691988] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2779.243168] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2787.592954] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2793.845436] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2801.278251] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2808.224246] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2816.217605] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2820.735951] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2820.919261] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2825.231698] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2825.454704] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2832.686945] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2832.820964] Lustre: lustre-OST0001: new disk, initializing [ 2832.823607] Lustre: Skipped 2 previous similar messages [ 2832.826708] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2832.830687] Lustre: Skipped 3 previous similar messages [ 2832.834786] Lustre: 98661:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2840.845823] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2842.778176] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2842.795822] Lustre: Skipped 2 previous similar messages [ 2842.802646] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2842.904898] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2852.106441] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2856.765369] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2856.975412] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2861.265304] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2861.467323] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2884.402500] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2912.219214] Lustre: Failing over lustre-MDT0000 [ 2912.248799] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -19 [ 2912.264517] LustreError: Skipped 1 previous similar message [ 2912.278759] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2912.299737] Lustre: Skipped 2 previous similar messages [ 2912.307659] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2912.312388] Lustre: Skipped 8 previous similar messages [ 2912.990616] Lustre: server umount lustre-MDT0000 complete [ 2914.802420] LustreError: 95289:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2914.854120] LustreError: 95289:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 2931.167544] Lustre: 66685:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789778526/real 1789778526] req@000000002b3ec859 x1876717784552576/t0(0) o400->MGC192.168.202.111@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1789778542 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2931.188478] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2941.850620] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2941.853716] LDISKFS-fs (dm-0): recovery complete [ 2941.861624] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2955.783770] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x424b3ca924c33db8 [ 2955.795762] Lustre: MGC192.168.202.111@tcp: Connection restored to 0@lo (at 0@lo) [ 2956.190256] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2960.788815] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2961.104149] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2961.108711] Lustre: 99916:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2961.401856] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2961.505929] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2961.841425] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2961.851903] Lustre: 101517:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2961.864578] Lustre: 101517:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2962.192572] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2962.209758] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2962.565147] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2962.573537] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2962.920634] Lustre: 101517:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2962.931632] Lustre: 101517:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 2 previous similar messages [ 2963.249676] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2963.261299] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2963.583177] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2963.592090] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2965.199971] Lustre: 101517:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2965.214874] Lustre: 101517:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2965.540485] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2965.561701] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2965.887128] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2965.900353] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2969.369676] Lustre: 101517:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2969.407658] Lustre: 101517:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2969.737214] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2969.752097] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2970.079295] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2970.085421] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2977.408096] Lustre: 101517:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2977.433211] Lustre: 101517:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 2977.772649] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2977.780519] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2978.105379] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2978.114065] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2993.665539] Lustre: 101517:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2993.685924] Lustre: 101517:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2994.016891] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2994.024075] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2994.344606] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2994.355651] LustreError: 101517:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 3021.087807] Lustre: 101517:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3021.096096] Lustre: 101517:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3021.112808] Lustre: 101517:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 1186cfa0-ae9a-4dbf-ad36-f8c8a7e1d6d1@192.168.202.11@tcp [ 3021.135768] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3021.144358] Lustre: 101517:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@000000008e0aa0fd x1876717764420096/t0(8589934783) o101->1186cfa0-ae9a-4dbf-ad36-f8c8a7e1d6d1@192.168.202.11@tcp:558/0 lens 592/0 e 0 to 0 dl 1789778643 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3021.188522] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3021.204470] Lustre: lustre-MDT0000: Denying connection for new client 1186cfa0-ae9a-4dbf-ad36-f8c8a7e1d6d1 (at 192.168.202.11@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3021.208747] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3021.270311] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3021.273209] Lustre: Skipped 2 previous similar messages [ 3021.284784] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3021.341484] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:81 to 0x280000401:577) [ 3021.343898] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 3037.678257] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3043.273076] Lustre: server umount lustre-OST0000 complete [ 3043.309301] LustreError: 99124:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3043.337530] LustreError: 99124:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 53 previous similar messages [ 3047.904470] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3047.908920] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 3047.930892] Lustre: Skipped 6 previous similar messages [ 3047.948822] Lustre: Skipped 6 previous similar messages [ 3053.843822] Lustre: server umount lustre-OST0001 complete [ 3065.282742] Lustre: server umount lustre-MDT0000 complete [ 3069.024876] LustreError: 95276:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789778680 with bad export cookie 4776978526920457656 [ 3069.031731] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3069.037596] LustreError: 95276:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3069.355841] Lustre: server umount lustre-MDT0001 complete [ 3079.346155] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 20:44:48 (1789778688) [ 3088.032725] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3088.043038] Lustre: *** cfs_fail_loc=197, val=0*** [ 3103.775639] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3108.898098] LustreError: 103491:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.111@tcp: failed processing log, type 4: rc = -110 [ 3114.050820] Lustre: *** cfs_fail_loc=197, val=0*** [ 3114.052766] Lustre: Skipped 65 previous similar messages [ 3134.623312] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3134.628114] Lustre: Skipped 4 previous similar messages [ 3141.515844] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3149.448869] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3152.238884] Lustre: server umount lustre-OST0000 complete [ 3162.857461] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 20:46:12 (1789778772) [ 3185.176515] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3190.710899] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3201.995388] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3206.707956] LustreError: 105142:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3206.719956] LustreError: 105142:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3206.725158] LustreError: 105142:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 21, retries 0, failed: rc = -5 [ 3207.123960] Lustre: server umount lustre-MDT0000 complete [ 3244.481903] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 3256.100552] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 3298.437484] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3298.829290] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3298.911718] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3299.104351] Lustre: lustre-MDT0000: new disk, initializing [ 3299.340665] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3304.876595] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3317.754655] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3317.909427] Lustre: 109318:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3318.302925] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3324.571785] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3335.831465] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3341.674872] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3349.305450] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3349.485223] Lustre: 111290:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3356.546131] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3359.424200] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3366.262738] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3370.902828] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3371.054225] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3376.657555] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3377.034199] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3380.195212] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3380.204027] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3380.206332] LustreError: Skipped 4 previous similar messages [ 3380.206990] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3380.206993] Lustre: Skipped 7 previous similar messages [ 3380.239085] Lustre: Skipped 3 previous similar messages [ 3385.317511] LustreError: 111275:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3385.334454] LustreError: 111275:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 11 previous similar messages [ 3385.503994] Lustre: server umount lustre-OST0000 complete [ 3396.718394] Lustre: server umount lustre-MDT0000 complete [ 3401.365819] LustreError: 109312:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789779013 with bad export cookie 4776978526920468380 [ 3401.386536] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3401.835599] Lustre: server umount lustre-MDT0001 complete [ 3412.493475] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 20:50:21 (1789779021) [ 3422.380569] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 3426.023582] Key type lgssc unregistered [ 3426.366060] LNet: 113777:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3426.378569] LNetError: 113777:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3426.408320] LNet: Removed LNI 192.168.202.111@tcp [ 3427.319653] Key type .llcrypt unregistered [ 3427.320924] Key type ._llcrypt unregistered [ 3453.362378] Key type ._llcrypt registered [ 3453.364809] Key type .llcrypt registered [ 3453.464042] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 3472.878736] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 3473.594549] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3473.625595] alg: No test for adler32 (adler32-zlib) [ 3474.714555] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [ 3474.911223] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 3476.559217] Key type lgssc registered [ 3477.712910] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3523.217817] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3523.264444] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3524.511526] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3524.534667] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3524.601048] Lustre: lustre-MDT0000: new disk, initializing [ 3524.673565] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3524.691189] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3529.164908] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3540.336199] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3540.672174] Lustre: lustre-OST0000: new disk, initializing [ 3540.677482] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3540.686312] Lustre: Skipped 1 previous similar message [ 3540.690305] Lustre: 117954:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3540.785635] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3548.008935] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3549.387238] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3549.403773] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3549.442559] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3561.359673] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3561.678143] Lustre: lustre-OST0001: new disk, initializing [ 3561.697119] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3561.721342] Lustre: 118817:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3561.900188] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3570.533370] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3571.824841] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3571.829328] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3571.875267] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3581.923648] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3581.938922] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3581.950537] Lustre: Skipped 1 previous similar message [ 3585.921652] Lustre: server umount lustre-MDT0000 complete [ 3593.074444] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3598.266333] LustreError: 117140:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789779209 with bad export cookie 11550015876534973419 [ 3598.283429] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3598.493864] Lustre: server umount lustre-OST0000 complete [ 3603.097539] Lustre: server umount lustre-OST0001 complete [ 3613.620168] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 20:53:42 (1789779222) [ 3638.491818] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 20:54:07 (1789779247) [ 3648.823393] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 3674.057471] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 20:54:43 (1789779283) [ 3714.722955] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 3727.151497] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 3769.279557] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3769.553838] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3769.577636] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3769.672326] Lustre: lustre-MDT0000: new disk, initializing [ 3769.743917] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3769.758624] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3775.143727] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3787.017458] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3787.177052] Lustre: 126575:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3787.263316] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3787.267312] Lustre: Skipped 1 previous similar message [ 3787.341473] Lustre: lustre-MDT0001: new disk, initializing [ 3787.428584] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3787.470842] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3787.486289] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3792.699361] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3802.653548] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3808.749933] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3816.248966] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3816.643060] Lustre: lustre-OST0000: new disk, initializing [ 3816.649102] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3816.658981] Lustre: 128546:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3816.778373] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3823.724856] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3825.767272] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3825.782873] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3825.934140] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3836.032266] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3840.978518] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3841.212296] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3846.318227] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3846.559832] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3851.755728] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3851.761838] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3851.779590] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3851.782511] Lustre: Skipped 1 previous similar message [ 3851.801221] Lustre: Skipped 2 previous similar messages [ 3855.204767] Lustre: server umount lustre-OST0000 complete [ 3859.425237] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3859.439914] LustreError: Skipped 1 previous similar message [ 3859.454826] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3859.489188] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3861.989369] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3861.990178] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3862.047646] Lustre: Skipped 1 previous similar message [ 3865.460951] Lustre: server umount lustre-MDT0000 complete [ 3869.309325] LustreError: 126568:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789779481 with bad export cookie 11550015876534974644 [ 3869.317369] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3869.330339] LustreError: 126568:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3869.685955] Lustre: server umount lustre-MDT0001 complete [ 3879.446648] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 20:58:08 (1789779488) [ 3887.914620] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3888.535799] LustreError: 130957:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3888.556228] LustreError: 130957:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3888.632296] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3893.731799] LustreError: 130958:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3894.318150] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3898.850592] LustreError: 130958:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3903.970571] LustreError: 130957:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3904.092190] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3909.534531] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3918.040216] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3924.870755] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3932.516881] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3932.951867] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3932.958811] Lustre: Skipped 1 previous similar message [ 3940.021830] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3948.202429] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3956.091410] Lustre: *** cfs_fail_loc=159, val=0*** [ 3956.098117] Lustre: Skipped 2 previous similar messages [ 3972.383659] Lustre: lustre-MDT0000: Client 796b1b86-8c19-4d29-81fb-372313136eab (at 192.168.202.11@tcp) reconnecting [ 3973.830630] Lustre: *** cfs_fail_loc=159, val=0*** [ 3973.832705] Lustre: Skipped 1 previous similar message [ 3989.313318] Lustre: lustre-MDT0000: Client 796b1b86-8c19-4d29-81fb-372313136eab (at 192.168.202.11@tcp) reconnecting [ 4010.979523] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4010.995213] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4011.034128] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4013.764519] Lustre: server umount lustre-OST0000 complete [ 4020.197799] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4020.201837] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4020.209089] Lustre: Skipped 2 previous similar messages [ 4020.219136] Lustre: Skipped 3 previous similar messages [ 4024.696523] Lustre: server umount lustre-MDT0000 complete [ 4025.312760] LustreError: 130952:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4025.345103] LustreError: 130952:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4030.485417] LustreError: 130939:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789779642 with bad export cookie 11550015876534975526 [ 4030.509881] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4030.950194] Lustre: server umount lustre-MDT0001 complete [ 4045.045176] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 4049.086529] Key type lgssc unregistered [ 4049.579115] LNet: 134737:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4049.583642] LNetError: 134737:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4049.610721] LNet: Removed LNI 192.168.202.111@tcp [ 4051.026092] Key type .llcrypt unregistered [ 4051.029691] Key type ._llcrypt unregistered [ 4081.106391] Key type ._llcrypt registered [ 4081.113277] Key type .llcrypt registered [ 4081.308732] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 4083.077402] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4083.172507] alg: No test for adler32 (adler32-zlib) [ 4084.270413] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [ 4084.508461] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 4086.247226] Key type lgssc registered [ 4087.896802] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4097.437231] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4097.469207] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4099.148404] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4103.699992] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4109.285926] LustreError: 136160:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4109.296242] LustreError: 136160:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4113.521677] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4113.766333] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4118.323648] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4126.769202] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4134.037640] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4141.511838] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4142.134846] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4145.211114] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4151.466996] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4164.653991] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4175.107454] Lustre: *** cfs_fail_loc=159, val=0*** [ 4191.523699] Lustre: lustre-MDT0000: Client bbdf772d-6d9c-4005-959a-f4b53a7c2bfd (at 192.168.202.11@tcp) reconnecting [ 4194.056390] Lustre: *** cfs_fail_loc=159, val=0*** [ 4210.463183] Lustre: lustre-MDT0001: Client bbdf772d-6d9c-4005-959a-f4b53a7c2bfd (at 192.168.202.11@tcp) reconnecting [ 4212.253260] Lustre: *** cfs_fail_loc=159, val=0*** [ 4212.257536] Lustre: Skipped 3 previous similar messages [ 4228.935683] Lustre: lustre-MDT0001: Client bbdf772d-6d9c-4005-959a-f4b53a7c2bfd (at 192.168.202.11@tcp) reconnecting [ 4234.323604] Lustre: 139204:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4237.280333] Lustre: *** cfs_fail_loc=159, val=0*** [ 4237.285450] Lustre: Skipped 5 previous similar messages [ 4253.901280] Lustre: lustre-MDT0000: Client 399a4295-d21d-4588-9ce0-f4a855da6ccb (at 192.168.202.11@tcp) reconnecting [ 4255.924420] Lustre: *** cfs_fail_loc=159, val=0*** [ 4255.940284] Lustre: Skipped 13 previous similar messages [ 4271.407372] Lustre: lustre-MDT0000: Client 399a4295-d21d-4588-9ce0-f4a855da6ccb (at 192.168.202.11@tcp) reconnecting [ 4273.785910] Lustre: 139497:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4279.803462] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 4279.804772] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4279.812553] LustreError: Skipped 1 previous similar message [ 4279.829498] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4281.315321] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4281.318335] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4281.345950] Lustre: Skipped 1 previous similar message [ 4285.927597] Lustre: server umount lustre-OST0000 complete [ 4291.552789] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4291.553355] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4291.556436] Lustre: Skipped 1 previous similar message [ 4296.184645] Lustre: server umount lustre-MDT0000 complete [ 4296.678499] LustreError: 139325:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4296.723328] LustreError: 139325:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4300.651422] LustreError: 138008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789779912 with bad export cookie 6787898727714927222 [ 4300.656685] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4300.665974] LustreError: 138008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4301.343072] Lustre: server umount lustre-MDT0001 complete [ 4314.555646] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 4317.008272] Key type lgssc unregistered [ 4317.240549] LNet: 140550:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4317.249794] LNetError: 140550:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4317.268549] LNet: Removed LNI 192.168.202.111@tcp [ 4318.282082] Key type .llcrypt unregistered [ 4318.284440] Key type ._llcrypt unregistered [ 4343.858481] Key type ._llcrypt registered [ 4343.860599] Key type .llcrypt registered [ 4343.994575] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 4344.891837] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4344.965303] alg: No test for adler32 (adler32-zlib) [ 4345.953161] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [ 4346.154898] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 4347.815194] Key type lgssc registered [ 4348.850610] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4356.926886] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4356.954249] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4358.534096] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4363.425037] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4368.865333] LustreError: 141961:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4368.881226] LustreError: 141961:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4371.681832] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4372.109057] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4376.770500] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4384.079636] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4391.557505] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4400.293070] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4400.854272] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4403.977423] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4408.453719] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4416.337959] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4417.769770] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4425.188430] Lustre: 144542:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4428.053579] Lustre: 144590:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4429.354883] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4430.306874] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4434.677832] Lustre: 144737:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4435.579921] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4436.627498] Lustre: 144836:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4442.082523] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4442.086916] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4442.096278] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4442.100706] Lustre: Skipped 1 previous similar message [ 4446.533446] Lustre: server umount lustre-OST0000 complete [ 4452.326357] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4452.330384] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4452.355053] Lustre: Skipped 2 previous similar messages [ 4452.380090] Lustre: Skipped 1 previous similar message [ 4456.424933] Lustre: server umount lustre-MDT0000 complete [ 4457.447053] LustreError: 141962:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4457.476593] LustreError: 141962:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4460.392587] LustreError: 141943:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789780072 with bad export cookie 13269689464915281516 [ 4460.400981] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4460.411324] LustreError: 141943:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4460.900877] Lustre: server umount lustre-MDT0001 complete [ 4473.419634] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 4476.684991] Key type lgssc unregistered [ 4477.096723] LNet: 145897:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4477.109049] LNetError: 145897:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4477.123918] LNet: Removed LNI 192.168.202.111@tcp [ 4478.167823] Key type .llcrypt unregistered [ 4478.171431] Key type ._llcrypt unregistered [ 4510.444519] Key type ._llcrypt registered [ 4510.447778] Key type .llcrypt registered [ 4510.557970] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 4511.869890] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4511.933199] alg: No test for adler32 (adler32-zlib) [ 4512.978662] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [ 4513.356044] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 4515.129987] Key type lgssc registered [ 4516.237234] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4525.244279] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4525.283593] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4527.155379] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4532.384281] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4536.301609] LustreError: 147310:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4536.316816] LustreError: 147310:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4541.413018] LustreError: 147311:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4542.042431] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4542.460302] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4547.666590] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4556.155714] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4562.539311] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4569.775202] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4570.306175] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4571.318681] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4577.344495] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4585.568334] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4587.783858] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4592.888381] Lustre: *** cfs_fail_loc=159, val=0*** [ 4592.894350] Lustre: Skipped 1 previous similar message [ 4608.273179] Lustre: lustre-MDT0000: Client 543bddd6-22d2-4e31-9516-91de2232f824 (at 192.168.202.11@tcp) reconnecting [ 4627.941870] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4627.965068] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4627.988371] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4628.964144] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4628.966866] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4628.968095] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4628.994088] Lustre: Skipped 1 previous similar message [ 4632.285113] Lustre: server umount lustre-OST0000 complete [ 4639.205788] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4639.213098] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4639.229361] Lustre: Skipped 1 previous similar message [ 4639.254099] Lustre: Skipped 1 previous similar message [ 4643.389684] Lustre: server umount lustre-MDT0000 complete [ 4644.323372] LustreError: 148043:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4644.340073] LustreError: 148043:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4647.071390] LustreError: 149165:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789780258 with bad export cookie 12426650825726850246 [ 4647.079530] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4647.085632] LustreError: 149165:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4647.398659] Lustre: server umount lustre-MDT0001 complete [ 4659.907144] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 4662.435609] Key type lgssc unregistered [ 4662.844725] LNet: 150977:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4662.861590] LNetError: 150977:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4662.901403] LNet: Removed LNI 192.168.202.111@tcp [ 4664.057728] Key type .llcrypt unregistered [ 4664.060435] Key type ._llcrypt unregistered [ 4692.664543] Key type ._llcrypt registered [ 4692.668294] Key type .llcrypt registered [ 4692.807062] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 4693.959369] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4694.105598] alg: No test for adler32 (adler32-zlib) [ 4695.235705] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [ 4695.487340] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 4697.159207] Key type lgssc registered [ 4698.325249] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4706.407480] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4706.426883] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4707.968538] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4712.030514] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4717.536685] LustreError: 152390:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4717.553300] LustreError: 152390:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4720.236855] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4720.694135] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4725.509436] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4734.642635] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4741.658728] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4750.390536] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4750.938125] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4753.025149] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4757.750780] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4765.304067] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4765.547806] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4786.415292] Lustre: lustre-MDT0001: evicting 52b4d076-291d-42e3-bb7f-d294f5646bfb (at 192.168.202.11@tcp) by administrative request [ 4786.431837] Lustre: Skipped 1 previous similar message [ 4819.936492] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4819.948639] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4819.977397] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4821.987279] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4821.990083] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4822.010049] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4823.921543] Lustre: server umount lustre-OST0000 complete [ 4832.225525] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4832.269686] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4832.283721] Lustre: Skipped 2 previous similar messages [ 4837.099277] Lustre: server umount lustre-MDT0000 complete [ 4837.380780] LustreError: 152387:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4837.411196] LustreError: 152387:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4841.512201] LustreError: 152391:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4841.550521] LustreError: 152391:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4841.779381] LustreError: 152370:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789780453 with bad export cookie 15986074457230570693 [ 4841.787883] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4841.795214] LustreError: 152370:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4842.174404] Lustre: server umount lustre-MDT0001 complete [ 4855.530571] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 4858.509835] Key type lgssc unregistered [ 4858.768740] LNet: 156458:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4858.773854] LNetError: 156458:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4858.786535] LNet: Removed LNI 192.168.202.111@tcp [ 4859.579739] Key type .llcrypt unregistered [ 4859.583484] Key type ._llcrypt unregistered [ 4946.794461] Key type ._llcrypt registered [ 4946.797221] Key type .llcrypt registered [ 4946.972122] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 4948.469576] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4948.524597] alg: No test for adler32 (adler32-zlib) [ 4949.645473] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [ 4949.845171] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 4951.489558] Key type lgssc registered [ 4952.982947] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4962.537738] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4962.574374] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4964.309897] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4968.752269] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4973.551076] LustreError: 158945:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4973.575692] LustreError: 158945:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4978.665813] LustreError: 158946:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4979.195541] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4979.764863] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4985.441981] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4994.394285] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5001.638434] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5011.338957] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5011.929486] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5021.195450] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5026.325750] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5031.678632] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5033.458408] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5046.754330] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5046.760414] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5046.770834] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5048.804336] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5048.808638] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5052.321195] Lustre: server umount lustre-OST0000 complete [ 5056.480930] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5056.489735] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5056.512543] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5056.524094] Lustre: Skipped 1 previous similar message [ 5059.040970] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5059.050129] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5062.585398] Lustre: server umount lustre-MDT0000 complete [ 5064.164690] LustreError: 158945:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5064.197994] LustreError: 158945:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5066.286911] LustreError: 159684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789780678 with bad export cookie 11956331413981178495 [ 5066.291428] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5066.299473] LustreError: 159684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5066.683355] Lustre: server umount lustre-MDT0001 complete [ 5077.336376] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 5080.301531] Key type lgssc unregistered [ 5080.693137] LNet: 162507:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5080.705266] LNetError: 162507:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5080.729299] LNet: Removed LNI 192.168.202.111@tcp [ 5081.703054] Key type .llcrypt unregistered [ 5081.704884] Key type ._llcrypt unregistered [ 5106.558190] Key type ._llcrypt registered [ 5106.563650] Key type .llcrypt registered [ 5106.707880] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 5107.931261] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5108.012237] alg: No test for adler32 (adler32-zlib) [ 5109.093807] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [ 5109.374970] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 5111.175237] Key type lgssc registered [ 5112.419601] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5127.099957] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 21:18:56 (1789780736) [ 5143.379724] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 5153.923994] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 5195.349174] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5195.388383] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5196.903338] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5196.975188] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5197.067650] Lustre: lustre-MDT0000: new disk, initializing [ 5197.134139] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5197.158833] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5203.693800] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5216.964627] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5217.112202] Lustre: 166769:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5217.180906] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5217.188140] Lustre: Skipped 1 previous similar message [ 5217.305940] Lustre: lustre-MDT0001: new disk, initializing [ 5217.389924] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5217.421203] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5217.429806] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5221.892552] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5231.516462] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5237.400146] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5244.189202] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5244.483250] Lustre: lustre-OST0000: new disk, initializing [ 5244.486782] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5244.494204] Lustre: 168738:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5244.604599] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5250.103832] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5250.119427] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5250.233586] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5250.838794] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5262.185606] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5276.129387] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5276.155488] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5276.157483] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5276.167166] Lustre: Skipped 1 previous similar message [ 5279.789350] Lustre: server umount lustre-OST0000 complete [ 5286.370927] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5286.377455] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5286.403520] Lustre: Skipped 2 previous similar messages [ 5286.439756] Lustre: Skipped 1 previous similar message [ 5290.908666] Lustre: server umount lustre-MDT0000 complete [ 5294.733857] LustreError: 166762:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789780906 with bad export cookie 9746611256432337990 [ 5294.736249] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5294.754833] LustreError: 166762:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5295.190074] Lustre: server umount lustre-MDT0001 complete [ 5307.677306] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 5310.150928] Key type lgssc unregistered [ 5310.438304] LNet: 170642:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5310.443798] LNetError: 170642:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5310.490726] LNet: Removed LNI 192.168.202.111@tcp [ 5311.464164] Key type .llcrypt unregistered [ 5311.467565] Key type ._llcrypt unregistered [ 5339.574422] Key type ._llcrypt registered [ 5339.577430] Key type .llcrypt registered [ 5339.695366] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 5341.238170] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5341.359882] alg: No test for adler32 (adler32-zlib) [ 5342.509058] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [ 5342.896772] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 5344.623242] Key type lgssc registered [ 5345.960198] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5355.423107] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5355.454906] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5356.882615] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5361.849400] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5367.269922] LustreError: 172057:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5367.307748] LustreError: 172057:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5371.392396] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5371.834686] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5376.564676] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5383.779289] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5389.298471] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5395.140972] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5395.594286] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5401.763194] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5402.437877] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5409.904592] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5417.398799] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5417.426827] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5417.453821] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5417.460171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.478057] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5418.411031] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5418.449752] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5418.462094] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5418.476332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.497546] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5419.672646] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5419.703241] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5419.715659] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5419.720623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.730174] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5420.739863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.743274] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5421.753288] Lustre: Skipped 1 previous similar message [ 5421.779179] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5421.797881] Lustre: Skipped 1 previous similar message [ 5421.809302] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5421.828758] Lustre: Skipped 1 previous similar message [ 5421.834620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.855758] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5421.860722] Lustre: Skipped 1 previous similar message [ 5422.807616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.647912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.571074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.367899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.492408] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5426.502574] Lustre: Skipped 4 previous similar messages [ 5426.516165] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5426.527255] Lustre: Skipped 4 previous similar messages [ 5426.530994] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5426.538416] Lustre: Skipped 4 previous similar messages [ 5426.541646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.549145] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5426.560078] Lustre: Skipped 4 previous similar messages [ 5427.429317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.444282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.390524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.372913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.209459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.287890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.566572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.943315] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5434.956143] Lustre: Skipped 7 previous similar messages [ 5434.990459] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5435.010841] Lustre: Skipped 7 previous similar messages [ 5435.024274] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5435.034789] Lustre: Skipped 7 previous similar messages [ 5435.053524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.072453] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5435.075439] Lustre: Skipped 7 previous similar messages [ 5436.290782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.313563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.486989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.551070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.797177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.598158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.573482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.824510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.854164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.905977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.081876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.050052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.342194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.473030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.631901] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5451.641876] Lustre: Skipped 14 previous similar messages [ 5451.680433] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5451.690504] Lustre: Skipped 14 previous similar messages [ 5451.695170] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5451.701891] Lustre: Skipped 14 previous similar messages [ 5451.705901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.715025] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5451.739645] Lustre: Skipped 14 previous similar messages [ 5452.883490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.759043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.064090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.263205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.276310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.484744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.306461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.499376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.713498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.013398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.138631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.120127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.275414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.193940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.199897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.044833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.032607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.881589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.873762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.673816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.718879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.492732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.389446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.246988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.097584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.969043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.950659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.686232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.529357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.163124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.999701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.958865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.951594] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5483.953602] Lustre: Skipped 32 previous similar messages [ 5483.962508] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5483.971620] Lustre: Skipped 32 previous similar messages [ 5483.976822] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5483.983192] Lustre: Skipped 32 previous similar messages [ 5483.990141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.999927] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5484.002901] Lustre: Skipped 32 previous similar messages [ 5484.784090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.720760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.577891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.644342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.581785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.425230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.213373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.130112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.952182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.064658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.160057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.236555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.226701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.063680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.062411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.123854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.033971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.129416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.967660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.122116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.952580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.000136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.975699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.917724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.931172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.057075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.046383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.131594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.974486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.111538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.215492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.062290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.990418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.921909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.937878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.024344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.911523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.701364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.705588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.732136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.707887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.541727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.297165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.236661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.114137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.096919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.843383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.642471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.484858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.383384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.296342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.228077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.062119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.966686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.884570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.801405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.791811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.740477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.765765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.798482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.766711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.821800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.928743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.070145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.131643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.007586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.873891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.934525] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5548.948444] Lustre: Skipped 67 previous similar messages [ 5548.980598] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5549.002933] Lustre: Skipped 67 previous similar messages [ 5549.021846] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5549.029743] Lustre: Skipped 67 previous similar messages [ 5549.034994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.052247] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5549.059761] Lustre: Skipped 67 previous similar messages [ 5550.109822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.015420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.942729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.681361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.435871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.230339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.256197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.183857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.295294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.324827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.281365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.307946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.367331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.442623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.318472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.324273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.301482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.207203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.947486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.791639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.466675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.351427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.117821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.035775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.818831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.663027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.682294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.752315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.591991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.434734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.364235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.481799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.450101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.700825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.725525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.797606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.893793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.111427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.087355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.833235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.687796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.893666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.785395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.142416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.313559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.543643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.559372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.485129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.737211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.652642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.840506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.839257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.022976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.218215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.269990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.395937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.769348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.860164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.024298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.181257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.130516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.095185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.067692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.167243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.353920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.162765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.096819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.758628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.554686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.252713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.275937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.476173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.691587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.774482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.839380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.871438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.035246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.960606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.823970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.810793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.079866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.429536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.676936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.639662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.099520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.098774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.092541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.223227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.802161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.041758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.874536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.798205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.862266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.930116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.004247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.172250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.269689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.446347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.508412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.455314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.593297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.584744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.560476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.562707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.474311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.462848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.489445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.534646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.299988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.092776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.021098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.839683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.798359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.773534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.619584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.612193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.479541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.322203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.420350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.743069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.009290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.199951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.385484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.442221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.584821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.754031] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5677.759131] Lustre: Skipped 125 previous similar messages [ 5677.802730] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5677.826421] Lustre: Skipped 125 previous similar messages [ 5677.832662] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5677.839584] Lustre: Skipped 125 previous similar messages [ 5677.849566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.867989] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5677.880409] Lustre: Skipped 125 previous similar messages [ 5678.687811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.605766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.694557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.643560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.694855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.532550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.448552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.424745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.238787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.063578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.919679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.009847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.112222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.120825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.176393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.054852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.355448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.542931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.723909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.894588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.081873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.208950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.422559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.905861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.879819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.785437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.779253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.672118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.996437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.101361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.403147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.400393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.418662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.457311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.615395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.799938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.232173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.532527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.934834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.972690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.924786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.911599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.983559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.802526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.957200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.259256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.896810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.225268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.898756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.462340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.529399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.653455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.051662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.020166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.243801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.170135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.387156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.851872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.117972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.357303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.774965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.862676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.016354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.971591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.289162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.279722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.283774] LustreError: 172077:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5753.296125] LustreError: 172077:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5753.315232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.174449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.187596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.082123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.298570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.182645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.144297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.923398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.903813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.969685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.687614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.526912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.411505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.284474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.201576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.066806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.062621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.021477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.039881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.112470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.945987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.997397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.263181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.167526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.179598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.825688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.696185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.625647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.608684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.482409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.582292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.531892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.436232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.297514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.458141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.367763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.320571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.184169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.303663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.322815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.258701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.233579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.082548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.279099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.356272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.437863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.314727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.299132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.092612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.270432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.396501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.768693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.944125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.871655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.924502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.205729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.210569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.276423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.485165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.587511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.069846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.421091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.427531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.808620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.870986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.860124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.985798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.088240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.330514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.424457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.547993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.577980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.476552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.303306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.280472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.265128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.325876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.131450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.087454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.055772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.056584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.121323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.245787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.140917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.999146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.189793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.153147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.993084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.034381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.365545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.558756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.583591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.540097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.586200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.612369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.515978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.310931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.351060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.260039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.326983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.246855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.326061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.468022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.703371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.599651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.771637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.734585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.857301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.831843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.000995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.089593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.218641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.313751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.371857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.359151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.454179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.524420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.740719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.999188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.166494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.136202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.397415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.598911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.768832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.006832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.098081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.215056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.498834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.625693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.872559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.035887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.189228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.396621] LustreError: 172077:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5890.412946] LustreError: 172077:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5890.427778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.612296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.805835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.884793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.137616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.350906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.253772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.470169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.633911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.507964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.887445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.898701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.262058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.279564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.658667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.948438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.072145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.046652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.295139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.431615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.361890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.609486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.896269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.904830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.745988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.739734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.846071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.887892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.351730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.888804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.353738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.704360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.926992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.405802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.898553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.339988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.313597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.710415] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5934.716875] Lustre: Skipped 235 previous similar messages [ 5934.748162] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5934.762986] Lustre: Skipped 235 previous similar messages [ 5934.789272] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5934.801392] Lustre: Skipped 235 previous similar messages [ 5934.812036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.840103] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5934.851977] Lustre: Skipped 235 previous similar messages [ 5936.186507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.240222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.383402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.399845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.618202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.024366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.082294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.055710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.380203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.381948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.481906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.591366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.766356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.955849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.013545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.297207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.456192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.821724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.328116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.574425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.001842] LustreError: 172077:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5960.016292] LustreError: 172077:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5960.046102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.226370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.216590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.380010] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.748895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.787945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.039875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.254957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.618408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.063325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.183527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.196652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.429164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.402638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.458638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.797919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.945185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.313099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.391142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.681801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.962966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.081897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.198614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.297410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.662583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.058041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.370932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.272414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.385863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.163237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.158484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.914801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.943733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.971737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.576183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.786088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.064940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.136577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.433885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.458202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.394208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.331062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.336284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.420226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.612553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.616678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.646976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.624602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.736777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.048492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.052859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.181664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.139365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.426307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.621595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.538692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.601144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.720890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.625785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.864394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.646977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.608050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.477482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.673661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.820112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.840242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.962661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.003027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.911343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.993548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.972538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.740411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.548119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.340900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.209334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.264786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.540294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.594127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.469419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.536547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.625846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.519847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.487817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.323877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.537921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.612554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.654622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.577800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.714089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.782454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.784209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.797275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.934256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.863307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.996782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.971829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.127512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.183988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.327760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.521399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.602295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.984256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.966659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.197290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.282827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.423583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.707808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.580308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.789670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.662377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.840960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.817883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.872611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.735995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.972589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.190232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.164767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.170605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.068674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.231697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.278097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.285502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.488421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.048587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.418522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.782369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.060103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.706506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.091724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.359933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.468213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.512930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.559967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.575843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.764531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.039807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.924787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.313511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.455143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.529851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.642678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.858412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.971373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.102579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.192290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.036654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.283624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.287378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.393711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.261801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.235938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.185050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.186593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.417858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.512768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.559405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.628787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.673257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.816474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.987598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.410777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.665102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.790584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.839663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.044094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.994780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.138383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.341841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.364157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.606858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.623657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.513411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.715440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.787535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.057888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.024232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.252291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.568611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.690601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.284781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.753548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.305444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.795286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.155161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.619658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.746698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.146106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.528959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.892700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.134122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.362823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.768593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.801642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.128718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.320419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.300590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.493690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.657044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.167756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.795624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.926454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.067345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.068219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.992866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.033137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.987032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.019462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.186801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.993962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.977872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.982810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.868085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.877359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.933691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.024207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.128869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.056817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.033699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.935649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.978760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.909941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.036247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.867424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.874503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.999662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.243635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.343677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.568792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.689190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.720662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.704563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.722681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.788605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.592139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.847703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.837250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.997189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.054496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.145803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.341741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.279642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.356882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.365052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.457742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.782167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.948924] LustreError: 172077:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6232.965330] LustreError: 172077:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6232.975454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.186325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.174483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.347860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.434099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.688293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.750339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.818692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.901532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.163850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.117050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.340358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.312641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.545204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.830020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.984327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.177658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.197221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.221138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.228350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.301628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.491867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.544849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.716982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.928186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.081990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.224932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.264910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.501292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.015286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.602643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.849244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.840726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.019413] LustreError: 172077:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6271.031882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.045253] LustreError: 172077:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6272.472555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.896278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.143769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.314175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.289575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.410562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.363137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.834812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.183461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.510539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.735366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.950890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.007724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.018860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.079439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.206932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.433704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.325943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.720083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.988638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.420153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.581125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.323030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.253845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.057695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.955241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.791779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.105930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.262996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.253692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.355904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.303597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.343876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.342802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.507099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.586627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.664152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.815127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.111908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.142482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.199903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.383498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.492934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.662568] LustreError: 172077:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6320.675213] LustreError: 172077:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6320.683345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.804808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.814521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.947941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.922218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.212465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.395586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.492755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.675449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.799455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.927020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.012549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.150671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.352500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.450129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.438724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.571796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.544883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.712176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.696540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.784865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.822685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.760665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.774156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.740402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.704879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.810297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.758701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.967366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.161240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.261063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.365497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.469740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.724325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.748514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.331836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.618919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.498807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.615621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.968110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.946420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.114787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.788565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.809976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.925679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.204357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.509459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.679397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.605907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.580864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.375255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.440337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.326145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.363160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.351567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.325370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.349678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.366138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.334558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.577892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.418367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.998676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.348321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.829681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.248788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.738836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.250803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.912327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.123466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.423681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.705301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.021924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.145906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.391092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.889102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.035309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.238727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.528605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.022766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.444958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.570255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.623223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.487946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.459524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.546841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.588783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.647701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.557247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.699504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.607821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.665780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.529142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.346314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.384150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.429875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.570757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.637565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.646397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.864881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.878422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.043103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.880602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.039237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.959536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.005629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.033332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.166667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.521989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.491062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.644337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.661763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.802439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.804471] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6446.813317] Lustre: Skipped 454 previous similar messages [ 6446.834653] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6446.847917] Lustre: Skipped 454 previous similar messages [ 6446.854604] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6446.860268] Lustre: Skipped 454 previous similar messages [ 6446.868995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.882222] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6446.896293] Lustre: Skipped 454 previous similar messages [ 6447.988673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.872954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.005287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.075842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.944650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.057395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.202223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.195935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.307789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.262781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.325395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.225481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.367540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.502495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.841627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.892983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.968432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.178535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.228406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.310956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.609897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.614975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.642166] LustreError: 172077:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6471.656493] LustreError: 172077:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6471.662288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.483527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.546334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.618970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.589401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.702272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.712257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.816667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.970512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.130932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.088884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.107343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.326321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.329433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6486.753692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.300432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.733052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.361445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.755551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.123229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.496081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.626759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.002834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.399164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.888906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.057058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.363153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6992.779469] LustreError: 174187:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6992.794475] LustreError: 172077:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6992.799413] LustreError: 172077:0:(osp_precreate.c:655:osp_precreate_send()) Skipped 2 previous similar messages [ 7757.290320] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7757.302834] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7757.328626] Lustre: Skipped 49 previous similar messages [ 7757.332528] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7757.838237] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 7757.843833] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7757.851223] LustreError: Skipped 3 previous similar messages [ 7758.243995] Lustre: server umount lustre-OST0000 complete [ 7762.912095] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7762.922398] LustreError: Skipped 2 previous similar messages [ 7762.928629] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7768.576078] LustreError: 172058:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7768.594343] LustreError: 172058:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7768.890773] Lustre: server umount lustre-MDT0000 complete [ 7772.934915] LustreError: 172039:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789783384 with bad export cookie 9628244520594499069 [ 7772.939272] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7772.950258] LustreError: 172039:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7773.339548] Lustre: server umount lustre-MDT0001 complete [ 7787.735708] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 7791.534255] Key type lgssc unregistered [ 7792.175382] LNet: 227374:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7792.187926] LNetError: 227374:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7792.221483] LNet: Removed LNI 192.168.202.111@tcp [ 7793.707518] Key type .llcrypt unregistered [ 7793.723288] Key type ._llcrypt unregistered [ 7823.985965] Key type ._llcrypt registered [ 7823.988453] Key type .llcrypt registered [ 7824.112537] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 7825.104293] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7825.290375] alg: No test for adler32 (adler32-zlib) [ 7826.560808] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [ 7826.884942] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 7828.615328] Key type lgssc registered [ 7830.101335] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7839.045858] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7839.086730] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7840.615906] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7845.576802] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7850.980877] LustreError: 228785:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7851.009113] LustreError: 228785:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7854.893065] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7855.279460] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7860.054170] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7869.104127] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7876.112259] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7884.565780] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7885.026737] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7886.119818] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7893.619256] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7904.684066] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7911.815801] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7913.399568] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7913.409734] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7913.417856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7913.436855] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7913.438077] LustreError: 228804:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7918.310502] Lustre: lustre-OST0000: Not available for connect from 192.168.202.11@tcp (stopping) [ 7920.100205] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7920.102054] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7920.124939] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7921.934892] Lustre: server umount lustre-OST0000 complete [ 7933.519613] LustreError: 228804:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7958.774276] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7959.283263] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7960.309298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7960.320804] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7960.327813] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7960.341606] Lustre: Skipped 1 previous similar message [ 7966.244189] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7975.545192] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7986.250250] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 8 sec [ 7993.176830] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8e214687c000.ost_server_uuid 50 [ 7995.163894] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8e214687c000.ost_server_uuid in FULL state after 0 sec [ 8000.480592] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8000.490197] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8000.501489] Lustre: Skipped 1 previous similar message [ 8000.507799] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8000.510661] Lustre: Skipped 1 previous similar message [ 8001.543834] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8002.477592] Lustre: server umount lustre-OST0000 complete [ 8008.673637] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8008.678837] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8008.687717] Lustre: Skipped 1 previous similar message [ 8008.694964] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8008.700810] Lustre: Skipped 2 previous similar messages [ 8013.411228] Lustre: server umount lustre-MDT0000 complete [ 8016.872922] LustreError: 228780:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8016.944812] LustreError: 228780:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8018.259595] LustreError: 228765:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789783629 with bad export cookie 3943424569423000077 [ 8018.262350] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8018.272597] LustreError: 228765:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8018.676964] Lustre: server umount lustre-MDT0001 complete [ 8033.957317] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 8037.489259] Key type lgssc unregistered [ 8037.886770] LNet: 233996:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8037.902545] LNetError: 233996:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8037.922738] LNet: Removed LNI 192.168.202.111@tcp [ 8039.299095] Key type .llcrypt unregistered [ 8039.302858] Key type ._llcrypt unregistered [ 8081.493990] Key type ._llcrypt registered [ 8081.501899] Key type .llcrypt registered [ 8081.645638] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 8083.051532] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8083.213313] alg: No test for adler32 (adler32-zlib) [ 8084.244621] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [ 8084.610967] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 8086.368235] Key type lgssc registered [ 8088.200459] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8101.306070] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8101.333180] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8103.059788] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8109.683188] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8112.611483] LustreError: 235728:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8112.631139] LustreError: 235728:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8117.735155] LustreError: 235727:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8119.510412] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8120.107763] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8124.859609] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8127.853627] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8135.474987] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8135.811489] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8144.002773] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8148.077616] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 8155.408801] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8155.612286] Lustre: lustre-OST0001: new disk, initializing [ 8155.625422] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8155.639127] Lustre: 238035:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8155.827540] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8164.941176] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8166.090052] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8166.111474] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8166.354773] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8178.271627] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8185.075808] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8198.047467] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [ 8198.054188] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8198.072866] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8198.468955] Lustre: server umount lustre-MDT0000 complete [ 8202.226995] LustreError: 235727:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8202.237263] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8202.332516] Lustre: Skipped 1 previous similar message [ 8203.797956] LustreError: 235710:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789783815 with bad export cookie 10038964386382033095 [ 8203.807426] LustreError: 236464:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 8203.809973] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8203.813794] LustreError: 235710:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8204.580475] Lustre: server umount lustre-MDT0001 complete [ 8219.234382] Lustre: server umount lustre-OST0000 complete [ 8234.070375] Lustre: server umount lustre-OST0001 complete [ 8263.747931] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 8276.355842] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8276.941578] LustreError: 241159:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8276.981140] LustreError: 241159:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8277.140144] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8282.595492] LustreError: 241160:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8282.624895] LustreError: 241160:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8283.177831] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8291.819360] LustreError: 241159:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8291.852878] LustreError: 241159:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8293.824103] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8294.586748] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8300.985779] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8304.472290] Lustre: 242303:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8312.152198] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8312.642081] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8319.782593] LustreError: 242657:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8319.808128] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 8320.106374] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8321.857785] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 8330.541965] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8338.131212] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8348.061922] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8352.450163] Lustre: 244173:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8367.074163] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8367.077383] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8367.102501] Lustre: Skipped 4 previous similar messages [ 8367.118591] Lustre: Skipped 3 previous similar messages [ 8372.204788] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8372.212839] Lustre: Skipped 2 previous similar messages [ 8372.986337] Lustre: server umount mylustre-MDT0000 complete [ 8377.317115] LustreError: 244484:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8377.345651] LustreError: 244484:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 8377.452573] LustreError: 244176:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789783989 with bad export cookie 10038964386382035958 [ 8377.454391] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8377.459230] LustreError: 244176:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8377.756697] Lustre: server umount mylustre-MDT0001 complete [ 8392.092261] Lustre: server umount mylustre-OST0000 complete [ 8407.699469] Lustre: server umount mylustre-OST0001 complete [ 8439.270653] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 8451.398145] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8452.499824] LustreError: 246491:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8452.712337] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8452.725454] Lustre: Skipped 1 previous similar message [ 8459.435460] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8474.088431] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8480.814180] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8484.214626] Lustre: 247634:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8491.361841] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8497.776117] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 8499.836536] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:161) [ 8500.465667] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8516.046279] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8521.727019] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 8525.260576] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8536.442439] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8541.589405] Lustre: 249506:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8552.420556] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8552.451783] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8552.453713] Lustre: Skipped 1 previous similar message [ 8552.472946] Lustre: Skipped 3 previous similar messages [ 8557.541167] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8557.561966] Lustre: Skipped 3 previous similar messages [ 8558.636146] Lustre: server umount tfs-MDT0000 complete [ 8564.691104] LustreError: 246467:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789784176 with bad export cookie 10038964386382037911 [ 8564.720755] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8564.726675] LustreError: 246467:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 8565.784602] Lustre: server umount tfs-MDT0001 complete [ 8582.753872] Lustre: server umount tfs-OST0000 complete [ 8584.159130] Lustre: 234988:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789784179/real 1789784179] req@ffff922da44d2300 x1876724229044480/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789784195 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8584.202171] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8584.212669] Lustre: Skipped 2 previous similar messages [ 8585.701101] Lustre: 234989:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789784181/real 1789784181] req@ffff922d84e68380 x1876724229044736/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789784197 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8587.135976] Lustre: server umount tfs-OST0001 complete [ 8613.217752] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 8626.797926] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8627.292107] LustreError: 251759:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8627.318161] LustreError: 251759:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 17 previous similar messages [ 8627.438179] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8627.449787] Lustre: Skipped 3 previous similar messages [ 8633.280087] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8644.227868] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8650.065822] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8653.034578] Lustre: 252902:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8660.930316] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8668.763715] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8669.704937] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:193) [ 8673.792110] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 8678.798574] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8684.559768] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 8686.720977] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8697.377263] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8703.171603] Lustre: 254775:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8714.569551] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 22:18:43 (1789784323) [ 8733.668157] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8733.673163] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8733.692416] Lustre: Skipped 3 previous similar messages [ 8733.708940] Lustre: Skipped 3 previous similar messages [ 8739.516821] Lustre: server umount lustre-MDT0000 complete [ 8744.594611] LustreError: 251735:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789784356 with bad export cookie 10038964386382039829 [ 8744.600720] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8744.614671] LustreError: 251735:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8751.233496] Lustre: server umount lustre-MDT0001 complete [ 8754.775464] Lustre: server umount lustre-OST0000 complete [ 8758.669982] Lustre: server umount lustre-OST0001 complete [ 8766.432859] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 8778.144498] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 8832.756738] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8833.210210] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8833.302883] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8833.369443] Lustre: lustre-MDT0000: new disk, initializing [ 8833.474322] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8833.479657] Lustre: Skipped 3 previous similar messages [ 8833.498507] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8841.809968] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8853.838693] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8854.090476] Lustre: 257962:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8854.234365] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8854.244091] Lustre: Skipped 1 previous similar message [ 8854.377818] Lustre: lustre-MDT0001: new disk, initializing [ 8854.542824] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8854.556366] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8860.210533] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8874.987958] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8875.416708] Lustre: lustre-OST0000: new disk, initializing [ 8875.425586] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8875.437156] Lustre: 259545:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8877.286507] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8877.304698] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8877.544755] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8883.433277] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8896.124061] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8896.513911] Lustre: lustre-OST0001: new disk, initializing [ 8896.517341] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8896.522601] Lustre: 260416:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8897.792388] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8897.826426] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8897.938602] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8904.783315] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8919.725920] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 22:22:08 (1789784528) [ 8939.517987] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 22:22:28 (1789784548) [ 8959.467559] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8959.478752] Lustre: Skipped 3 previous similar messages [ 8959.480533] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8959.481817] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8959.490311] Lustre: Skipped 7 previous similar messages [ 8960.678324] Lustre: server umount lustre-OST0000 complete [ 8964.591526] LustreError: 259530:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8964.611953] LustreError: 259530:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 15 previous similar messages [ 8967.137114] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8967.149672] LustreError: Skipped 1 previous similar message [ 8971.181987] Lustre: server umount lustre-MDT0000 complete [ 8975.732701] LustreError: 257953:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789784587 with bad export cookie 10038964386382041607 [ 8975.745562] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8975.746441] LustreError: 257953:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8990.177286] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8990.808057] Lustre: server umount lustre-MDT0001 complete [ 9009.214581] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 9086.025392] Lustre: server umount lustre-OST0001 complete [ 9093.647494] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 9110.754795] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 9157.835168] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9158.158702] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9158.224732] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9158.383921] Lustre: lustre-MDT0000: new disk, initializing [ 9158.497682] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9158.507932] Lustre: Skipped 3 previous similar messages [ 9158.555643] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9166.090732] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9181.935717] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9182.040824] Lustre: 265291:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9182.269475] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9182.277101] Lustre: Skipped 1 previous similar message [ 9182.509960] Lustre: lustre-MDT0001: new disk, initializing [ 9182.898810] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9182.945969] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9190.710470] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9203.618925] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9210.386932] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9217.938136] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9218.402868] Lustre: lustre-OST0000: new disk, initializing [ 9218.406239] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9218.411121] Lustre: 267263:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9219.659829] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9219.686326] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9219.801860] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9225.843314] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9236.256563] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9245.664768] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9245.670522] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9245.674646] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9245.676145] Lustre: Skipped 5 previous similar messages [ 9245.693763] Lustre: Skipped 8 previous similar messages [ 9250.690091] Lustre: server umount lustre-OST0000 complete [ 9269.215190] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9269.550714] Lustre: server umount lustre-MDT0000 complete [ 9273.909174] LustreError: 266096:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789784885 with bad export cookie 10038964386382043077 [ 9273.922588] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9273.941418] LustreError: 266096:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9274.361508] Lustre: server umount lustre-MDT0001 complete [ 9290.645435] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 9295.584688] Key type lgssc unregistered [ 9295.925809] LNet: 269025:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9295.930294] LNetError: 269025:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9295.960099] LNet: Removed LNI 192.168.202.111@tcp [ 9296.969157] Key type .llcrypt unregistered [ 9296.972199] Key type ._llcrypt unregistered [ 9337.662590] Key type ._llcrypt registered [ 9337.667146] Key type .llcrypt registered [ 9337.805848] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 9339.088554] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9339.217077] alg: No test for adler32 (adler32-zlib) [ 9340.437843] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [ 9340.828957] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 9342.471197] Key type lgssc registered [ 9344.084054] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9352.233932] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9352.284158] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9354.109777] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9359.478216] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9363.426482] LustreError: 270487:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9363.448409] LustreError: 270487:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9368.545489] LustreError: 270488:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9371.814053] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9372.374859] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9378.440151] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9387.525597] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9393.019693] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9399.989065] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9400.435764] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9406.979286] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9414.343654] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9415.685612] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9422.308611] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9422.320507] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9422.352580] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9423.394130] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9424.672479] Lustre: server umount lustre-OST0000 complete [ 9433.568703] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9433.574558] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9433.583311] Lustre: Skipped 1 previous similar message [ 9433.606302] Lustre: Skipped 2 previous similar messages [ 9437.918489] Lustre: server umount lustre-MDT0000 complete [ 9438.705945] LustreError: 270487:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9443.249635] LustreError: 271223:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789785054 with bad export cookie 6751698707394494725 [ 9443.272153] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9443.281497] LustreError: 271223:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9443.865716] Lustre: server umount lustre-MDT0001 complete [ 9456.406570] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 9458.903570] Key type lgssc unregistered [ 9459.186820] LNet: 274081:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9459.191315] LNetError: 274081:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9459.206928] LNet: Removed LNI 192.168.202.111@tcp [ 9460.178657] Key type .llcrypt unregistered [ 9460.181110] Key type ._llcrypt unregistered [ 9478.200709] Key type ._llcrypt registered [ 9478.204183] Key type .llcrypt registered [ 9478.363501] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 9481.922119] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9481.943891] alg: No test for adler32 (adler32-zlib) [ 9482.910936] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [ 9483.151375] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 9484.967185] Key type lgssc registered [ 9488.046520] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9500.594694] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9500.625452] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9516.383393] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9521.503310] LustreError: 275270:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.111@tcp: failed processing log, type 4: rc = -110 [ 9548.127321] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9557.175318] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9566.469757] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9569.027761] Lustre: server umount lustre-OST0000 complete [ 9587.766301] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 9590.652239] Key type lgssc unregistered [ 9590.931482] LNet: 276901:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9590.943257] LNetError: 276901:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9591.998807] LNet: Removed LNI 192.168.202.111@tcp [ 9593.094789] Key type .llcrypt unregistered [ 9593.096540] Key type ._llcrypt unregistered [ 9644.602312] Key type ._llcrypt registered [ 9644.607434] Key type .llcrypt registered [ 9644.782259] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 9646.579794] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9646.698030] alg: No test for adler32 (adler32-zlib) [ 9647.853839] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [ 9648.213855] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 9649.967696] Key type lgssc registered [ 9651.462512] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9666.551954] loop0: detected capacity change from 0 to 204800000 [ 9666.836325] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9666.851901] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9667.300258] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9668.732455] loop0: detected capacity change from 0 to 204800000 [ 9668.987354] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9669.005770] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9669.478192] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9670.716804] loop0: detected capacity change from 0 to 204800000 [ 9670.931606] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9670.962988] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9671.078986] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9672.429028] loop0: detected capacity change from 0 to 204800000 [ 9672.763816] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9672.785081] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9672.961201] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9674.299225] loop0: detected capacity change from 0 to 204800000 [ 9674.327459] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9680.263253] loop0: detected capacity change from 0 to 204800000 [ 9680.334214] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9685.457539] loop0: detected capacity change from 0 to 204800000 [ 9685.501258] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9691.747365] loop0: detected capacity change from 0 to 204800000 [ 9691.786169] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9696.589067] loop0: detected capacity change from 0 to 204800000 [ 9696.709396] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9696.792189] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9696.862799] Lustre: 280313:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9696.878371] Lustre: 280313:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9696.887365] Lustre: 280313:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9696.897376] Lustre: 280313:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9696.913056] Lustre: 280313:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9696.923978] Lustre: 280313:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9696.934597] Lustre: 280313:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9696.947599] Lustre: 280313:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9696.959721] Lustre: 280313:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9696.974433] Lustre: 280313:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9696.986209] Lustre: 280313:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9696.996804] Lustre: 280313:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9697.011234] Lustre: 280313:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9697.025381] Lustre: 280313:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9703.504251] Lustre: server umount MGS complete [ 9704.839254] loop0: detected capacity change from 0 to 204800000 [ 9705.019442] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9705.710113] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9705.796777] Lustre: lustre-MDT0000: Aborting client recovery [ 9707.018570] loop1: detected capacity change from 0 to 204800000 [ 9707.249931] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9707.356148] Lustre: 280746:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9707.382348] Lustre: 280746:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9707.399038] Lustre: 280746:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9707.421842] Lustre: 280746:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9707.474865] Lustre: 280746:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9707.500057] Lustre: 280746:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9707.522730] Lustre: 280746:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9707.551923] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9707.879699] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9707.946730] Lustre: lustre-MDT0001: Aborting client recovery [ 9709.127729] loop2: detected capacity change from 0 to 204800000 [ 9709.315808] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9709.503538] Lustre: 280825:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9709.522776] Lustre: 280825:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9709.548079] Lustre: 280825:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9709.564433] Lustre: 280825:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9709.584261] Lustre: 280825:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9709.594784] Lustre: 280825:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9709.605471] Lustre: 280825:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9709.880089] Lustre: Found index 0 for lustre-OST0000, updating log [ 9710.022679] Lustre: 280844:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9710.178115] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9710.215518] Lustre: lustre-OST0000: Aborting recovery [ 9711.816438] loop3: detected capacity change from 0 to 204800000 [ 9712.097217] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9712.190388] Lustre: 280909:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9712.200651] Lustre: 280909:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9712.219979] Lustre: 280909:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9712.258515] Lustre: 280909:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9712.290650] Lustre: 280909:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9712.311851] Lustre: 280909:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9712.339208] Lustre: 280909:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9712.413921] Lustre: Found index 1 for lustre-OST0001, updating log [ 9712.503307] Lustre: 280916:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9712.615933] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9712.649592] Lustre: lustre-OST0001: Aborting recovery [ 9712.779349] LustreError: 280690:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 9715.693434] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9715.886851] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9715.916178] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9716.028400] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9717.798708] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9717.809577] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9723.110814] LustreError: 280656:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1133us: rc = -2 [ 9723.655756] LustreError: 280656:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0001: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0001 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0001/identity_upcall, time 769us: rc = -2 [ 9723.689203] LustreError: 280656:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 14 previous similar messages [ 9738.209941] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9738.216911] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9738.233828] Lustre: Skipped 2 previous similar messages [ 9738.250236] Lustre: Skipped 3 previous similar messages [ 9743.329230] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9743.337791] Lustre: Skipped 3 previous similar messages [ 9744.056744] Lustre: server umount lustre-MDT0000 complete [ 9745.235629] LustreError: 280638:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789785356 with bad export cookie 5999905007469848329 [ 9745.243181] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9745.252567] LustreError: 280638:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9748.448560] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9748.449180] LustreError: 281176:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9748.483455] Lustre: Skipped 1 previous similar message [ 9748.484490] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9748.540103] LustreError: 281176:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9750.495892] LustreError: 280653:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9750.496184] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9750.518270] Lustre: Skipped 2 previous similar messages [ 9751.860829] Lustre: server umount lustre-MDT0001 complete [ 9767.391730] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9767.807375] Lustre: server umount lustre-OST0000 complete [ 9775.759741] Lustre: server umount lustre-OST0001 complete [ 9787.287282] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 22:36:36 (1789785396) [ 9821.538669] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 9834.449886] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 9886.101689] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9886.477207] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9886.514155] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9886.594926] Lustre: lustre-MDT0000: new disk, initializing [ 9886.670655] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9886.689055] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9891.885743] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9905.019308] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9905.272706] Lustre: 284989:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9905.403786] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9905.414688] Lustre: Skipped 1 previous similar message [ 9905.590719] Lustre: lustre-MDT0001: new disk, initializing [ 9905.780475] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9905.835219] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9905.850747] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9911.079561] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9924.735657] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9932.774747] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9940.622079] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9940.825253] Lustre: lustre-OST0000: new disk, initializing [ 9940.826926] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9940.830169] Lustre: 286957:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9940.878974] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9948.011256] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9948.819090] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9948.830671] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9948.953563] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9958.347096] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9964.645871] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9974.646709] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9980.820561] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9988.578430] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10041.313563] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10041.324263] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10041.331471] LustreError: Skipped 1 previous similar message [10041.332146] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10041.409207] Lustre: Skipped 2 previous similar messages [10042.909523] Lustre: server umount lustre-OST0000 complete [10048.992497] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10049.004399] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10051.565168] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10051.573134] Lustre: Skipped 2 previous similar messages [10053.559037] Lustre: server umount lustre-MDT0000 complete [10056.677053] LustreError: 285001:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10056.713581] LustreError: 285001:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10058.292491] LustreError: 284982:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789785670 with bad export cookie 5999905007469850856 [10058.293868] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10058.308881] LustreError: 284982:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10058.808262] Lustre: server umount lustre-MDT0001 complete [10068.383328] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10073.374205] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10079.913850] Lustre: server umount MGS complete [10096.315234] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10097.033594] LustreError: 290781:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10097.081659] LustreError: 290794:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10097.170835] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10102.250724] LustreError: 290782:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10103.918179] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10111.470780] LustreError: 290781:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10111.486447] LustreError: 290781:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10116.113366] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10121.560835] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10129.954267] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10137.336563] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10146.531890] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10153.120414] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10162.386142] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10202.084964] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10202.100085] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10202.125399] Lustre: Skipped 1 previous similar message [10202.129260] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10202.138810] Lustre: Skipped 1 previous similar message [10205.885523] Lustre: server umount lustre-OST0000 complete [10212.320657] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10212.349509] Lustre: Skipped 1 previous similar message [10217.241685] Lustre: server umount lustre-MDT0000 complete [10217.446530] LustreError: 290782:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10217.467926] LustreError: 290782:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10222.822685] LustreError: 290762:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789785834 with bad export cookie 5999905007469852718 [10222.849035] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10222.854253] LustreError: 290762:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10223.279338] Lustre: server umount lustre-MDT0001 complete [10235.676698] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [10239.434032] Key type lgssc unregistered [10239.908329] LNet: 294728:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10239.930633] LNetError: 294728:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10239.959787] LNet: Removed LNI 192.168.202.111@tcp [10241.576900] Key type .llcrypt unregistered [10241.578864] Key type ._llcrypt unregistered [10287.144577] Key type ._llcrypt registered [10287.146257] Key type .llcrypt registered [10287.232038] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [10305.935990] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [10307.756475] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10307.794794] alg: No test for adler32 (adler32-zlib) [10308.958743] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [10309.423068] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [10311.200093] Key type lgssc registered [10312.877119] Lustre: Echo OBD driver; http://www.lustre.org/ [10353.358187] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10353.450530] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10354.867424] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10354.911926] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10355.005924] Lustre: lustre-MDT0000: new disk, initializing [10355.111819] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10355.137598] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10360.168438] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10372.291400] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10372.493859] Lustre: 298686:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10372.595614] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10372.609104] Lustre: Skipped 1 previous similar message [10372.782473] Lustre: lustre-MDT0001: new disk, initializing [10372.911772] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10372.971139] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10372.978761] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10377.702427] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10388.984182] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10395.477248] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10403.448481] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10403.875658] Lustre: lustre-OST0000: new disk, initializing [10403.888883] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10403.905953] Lustre: 300658:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10404.073548] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10410.569918] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10410.591428] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10410.742715] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10411.975953] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10423.149487] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10430.634964] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10441.491412] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10445.556225] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10450.831257] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10492.910930] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10492.915837] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10492.928773] LustreError: Skipped 1 previous similar message [10492.930272] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10492.980143] Lustre: Skipped 1 previous similar message [10498.028362] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10498.038896] Lustre: Skipped 1 previous similar message [10498.510069] Lustre: server umount lustre-OST0000 complete [10503.136402] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10503.138054] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10503.153226] Lustre: Skipped 1 previous similar message [10503.173593] Lustre: Skipped 1 previous similar message [10508.267180] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10508.277487] Lustre: Skipped 1 previous similar message [10509.677794] Lustre: server umount lustre-MDT0000 complete [10514.392241] LustreError: 298679:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789786126 with bad export cookie 6948882344672284357 [10514.401794] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10514.404150] LustreError: 298679:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10514.892088] Lustre: server umount lustre-MDT0001 complete [10524.848174] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10529.035542] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10534.095103] Lustre: server umount MGS complete [10545.714144] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10546.245403] LustreError: 304424:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10546.264502] LustreError: 304424:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10546.280368] LustreError: 304437:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10546.362996] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10551.564915] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10551.785914] LustreError: 304425:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10556.908114] LustreError: 304424:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10560.777421] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10561.223591] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10566.252299] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10575.185887] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10581.279446] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10589.943181] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10590.485079] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10598.569520] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10607.220690] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10641.888848] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10641.897361] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10641.900713] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10641.937769] Lustre: Skipped 1 previous similar message [10652.130184] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10652.133505] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 4. Is it stuck? [10652.141352] Lustre: Skipped 3 previous similar messages [10652.519831] Lustre: server umount lustre-OST0000 complete [10657.250617] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10657.260412] Lustre: Skipped 1 previous similar message [10663.227387] Lustre: server umount lustre-MDT0000 complete [10667.327505] LustreError: 304406:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789786279 with bad export cookie 6948882344672286198 [10667.344181] LustreError: 304406:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10667.346427] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10667.813995] Lustre: server umount lustre-MDT0001 complete [10681.340084] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [10684.422540] Key type lgssc unregistered [10684.635712] LNet: 308371:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10684.642114] LNetError: 308371:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10684.662311] LNet: Removed LNI 192.168.202.111@tcp [10685.538301] Key type .llcrypt unregistered [10685.540118] Key type ._llcrypt unregistered [10712.931133] Key type ._llcrypt registered [10712.932777] Key type .llcrypt registered [10713.097666] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [10729.987910] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [10731.324473] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10731.566484] alg: No test for adler32 (adler32-zlib) [10732.711922] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [10733.087257] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [10734.791519] Key type lgssc registered [10736.287647] Lustre: Echo OBD driver; http://www.lustre.org/ [10767.162324] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [10785.358535] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10785.372651] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10786.681386] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10786.716720] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10786.811947] Lustre: lustre-MDT0000: new disk, initializing [10786.885455] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10786.901184] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10791.891365] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10804.143896] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10804.433970] Lustre: lustre-OST0000: new disk, initializing [10804.443400] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10804.446927] Lustre: Skipped 1 previous similar message [10804.450649] Lustre: 312423:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10804.511586] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10810.994836] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10811.004721] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10811.052355] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10811.318919] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11727.332365] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11727.684561] Lustre: server umount lustre-MDT0000 complete [11731.718889] LustreError: 311610:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789787343 with bad export cookie 2542425834745490285 [11731.740903] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11731.846179] Lustre: server umount lustre-OST0000 complete [11760.318952] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [11772.367293] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [11811.605042] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11811.882809] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11811.906879] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11811.969341] Lustre: lustre-MDT0000: new disk, initializing [11812.058236] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11812.077397] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11817.818249] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11829.615971] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11829.825498] Lustre: 316785:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11829.883497] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11829.887661] Lustre: Skipped 1 previous similar message [11829.974515] Lustre: lustre-MDT0001: new disk, initializing [11830.104510] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11830.172246] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11830.192614] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11835.093920] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11844.999144] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11852.237739] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11860.684354] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11860.902983] Lustre: lustre-OST0000: new disk, initializing [11860.905963] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11860.910815] Lustre: 318755:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11860.980724] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11866.687970] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11866.710322] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11866.829065] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11867.710548] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11876.781117] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11881.040824] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11881.197332] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11885.987748] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11886.168874] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11892.705657] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11892.705995] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11892.718117] LustreError: Skipped 1 previous similar message [11892.722758] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11892.748342] Lustre: Skipped 1 previous similar message [11894.521987] Lustre: server umount lustre-OST0000 complete [11901.922222] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11901.942661] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11901.959868] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11901.963696] Lustre: Skipped 1 previous similar message [11904.586881] Lustre: server umount lustre-MDT0000 complete [11908.070412] LustreError: 316796:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11908.103445] LustreError: 316796:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11908.317980] LustreError: 316776:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789787520 with bad export cookie 2542425834748431104 [11908.322699] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11908.327706] LustreError: 316776:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11908.665146] Lustre: server umount lustre-MDT0001 complete [11912.453378] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11914.331625] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 23:12:03 (1789787523) [11920.221291] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11920.508491] LustreError: 321072:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11920.578710] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11925.228652] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11925.987982] LustreError: 321073:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11931.103994] LustreError: 321072:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11933.924493] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11934.417678] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11938.887760] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11946.985613] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11953.036576] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11959.887822] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11960.308840] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11966.233678] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11972.882611] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11979.136693] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11979.267918] Lustre: lustre-OST0001: new disk, initializing [11979.270349] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11979.276833] Lustre: 323907:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11979.375874] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11985.358621] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11987.579113] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11987.591916] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11987.678333] LustreError: 324367:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11988.710684] LustreError: 324367:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11988.725197] LustreError: 324367:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11989.816996] LustreError: 324371:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11989.841096] LustreError: 324371:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11991.926069] LustreError: 324367:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11991.944493] LustreError: 324367:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11995.364269] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11995.601515] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11996.072433] LustreError: 324367:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11996.100461] LustreError: 324367:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [12001.803964] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8e214e0ca000.ost_server_uuid 50 [12003.397624] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8e214e0ca000.ost_server_uuid in FULL state after 0 sec [12004.263404] LustreError: 324371:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12004.275467] LustreError: 324371:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [12009.692569] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8e214e0ca000.ost_server_uuid 50 [12011.101423] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8e214e0ca000.ost_server_uuid in FULL state after 0 sec [12012.568269] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [12026.595602] Lustre: lustre-OST0001: Not available for connect from 192.168.202.11@tcp (stopping) [12026.600902] Lustre: Skipped 2 previous similar messages [12027.874117] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12027.874948] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12027.878237] LustreError: Skipped 1 previous similar message [12027.895875] Lustre: Skipped 2 previous similar messages [12028.902698] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12028.907276] Lustre: Skipped 3 previous similar messages [12034.024794] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12034.042580] Lustre: Skipped 2 previous similar messages [12040.159205] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12040.482721] Lustre: server umount lustre-OST0001 complete [12041.936448] LustreError: 323196:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.202.11@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12049.380389] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12049.392181] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12049.393383] Lustre: Skipped 1 previous similar message [12049.403912] Lustre: Skipped 4 previous similar messages [12054.184402] Lustre: server umount lustre-OST0000 complete [12059.617711] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12064.349510] Lustre: server umount lustre-MDT0000 complete [12064.736078] LustreError: 321073:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12064.752423] LustreError: 321073:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [12068.661243] LustreError: 321052:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789787680 with bad export cookie 2542425834748431979 [12068.663214] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12068.674611] LustreError: 321052:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12069.314158] Lustre: server umount lustre-MDT0001 complete [12081.578982] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [12084.244911] Key type lgssc unregistered [12084.613157] LNet: 326595:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12084.617805] LNetError: 326595:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12084.641087] LNet: Removed LNI 192.168.202.111@tcp [12085.886714] Key type .llcrypt unregistered [12085.888422] Key type ._llcrypt unregistered [12115.402722] Key type ._llcrypt registered [12115.405834] Key type .llcrypt registered [12115.551560] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [12116.481260] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12116.499791] alg: No test for adler32 (adler32-zlib) [12117.654962] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [12117.871682] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [12119.703312] Key type lgssc registered [12120.825865] Lustre: Echo OBD driver; http://www.lustre.org/ [12127.692787] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12127.710346] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12129.168967] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12134.003859] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12138.469034] LustreError: 328014:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12138.480532] LustreError: 328014:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12141.169600] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12143.590427] LustreError: 328013:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12147.998245] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12148.410594] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12153.387328] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12163.597226] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12164.168968] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12171.202878] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12173.415504] LustreError: 329692:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12173.457271] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [12178.965106] LustreError: 330165:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12178.986379] LustreError: 330165:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12180.078622] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12184.047397] LustreError: 329691:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12184.073264] LustreError: 329691:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12187.049776] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12187.294161] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12192.775758] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [12193.669674] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12201.941981] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12210.382470] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [12210.766605] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12320.753712] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12320.753974] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12320.764209] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12320.772436] LustreError: Skipped 1 previous similar message [12320.791313] Lustre: Skipped 1 previous similar message [12324.320858] Lustre: server umount lustre-OST0001 complete [12325.867528] LustreError: 329691:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12325.880694] LustreError: 329691:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12332.512482] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12332.522287] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12332.530219] Lustre: Skipped 1 previous similar message [12332.534343] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12332.540379] Lustre: Skipped 1 previous similar message [12336.097302] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12336.115170] Lustre: Skipped 1 previous similar message [12337.448856] Lustre: server umount lustre-OST0000 complete [12342.753349] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12342.770718] LustreError: Skipped 1 previous similar message [12342.789159] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12342.810872] Lustre: Skipped 1 previous similar message [12342.830943] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12346.339216] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12348.697241] Lustre: server umount lustre-MDT0000 complete [12351.457923] LustreError: 328014:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12351.487957] LustreError: 328014:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [12352.861209] LustreError: 331698:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789787964 with bad export cookie 17059594148418570296 [12352.863268] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12352.876248] LustreError: 331698:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12353.496735] Lustre: server umount lustre-MDT0001 complete [12365.749637] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [12368.567190] Key type lgssc unregistered [12368.829636] LNet: 332955:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12368.835130] LNetError: 332955:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12368.847041] LNet: Removed LNI 192.168.202.111@tcp [12369.821728] Key type .llcrypt unregistered [12369.824131] Key type ._llcrypt unregistered [12418.144643] Key type ._llcrypt registered [12418.147070] Key type .llcrypt registered [12418.311532] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [12419.521725] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12419.576679] alg: No test for adler32 (adler32-zlib) [12420.669555] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [12421.006371] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [12422.807372] Key type lgssc registered [12423.996970] Lustre: Echo OBD driver; http://www.lustre.org/ [12433.850794] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12433.866645] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12435.478525] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12440.983952] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12445.674091] LustreError: 335243:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12445.694517] LustreError: 335243:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12450.529791] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12450.843562] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12455.937598] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12459.570455] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12466.737434] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12467.439516] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12475.268970] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12475.650499] LustreError: 336733:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12480.814417] LustreError: 336736:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12480.846432] LustreError: 336736:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12480.861765] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12486.125900] LustreError: 336736:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12486.160721] LustreError: 336736:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12487.345446] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12487.728085] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12492.802367] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12494.734262] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12505.328854] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12512.754634] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12527.584899] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12527.589805] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12527.607075] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12528.608493] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12528.622370] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12528.623511] Lustre: Skipped 2 previous similar messages [12528.652475] Lustre: Skipped 3 previous similar messages [12532.300494] Lustre: server umount lustre-MDT0000 complete [12533.737808] LustreError: 335243:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12533.755081] LustreError: 335243:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12536.637933] LustreError: 338314:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789788148 with bad export cookie 2665129459107733757 [12536.639308] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12536.650533] LustreError: 338314:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12537.048384] Lustre: server umount lustre-MDT0001 complete [12551.489899] Lustre: server umount lustre-OST0000 complete [12554.335595] Lustre: 334506:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789788150/real 1789788150] req@ffff922d8b677100 x1876728775509888/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789788166 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12554.365192] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12556.168730] Lustre: server umount lustre-OST0001 complete [12575.679437] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [12588.017199] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12588.362327] LustreError: 340512:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12588.433427] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12593.649427] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12604.383252] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12604.736392] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12609.593699] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12612.496242] Lustre: 341653:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12619.289472] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12625.843492] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12629.860843] LustreError: 342006:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12629.885283] LustreError: 342006:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12634.003795] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12636.365069] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12636.700901] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12636.708758] Lustre: Skipped 1 previous similar message [12641.775369] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12645.075386] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12655.089843] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12659.042245] Lustre: 343526:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12666.337182] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12666.346818] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12666.362662] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12669.435096] Lustre: server umount lustre-MDT0000 complete [12672.493948] LustreError: 341525:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12672.540643] LustreError: 341525:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [12673.675677] LustreError: 340494:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789788285 with bad export cookie 2665129459107735451 [12673.690719] LustreError: 340494:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12673.696696] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12674.053853] Lustre: server umount lustre-MDT0001 complete [12688.887722] Lustre: server umount lustre-OST0000 complete [12703.133559] Lustre: server umount lustre-OST0001 complete [12707.522509] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12709.267439] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12711.506829] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 23:25:20 (1789788320) [12744.594204] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [12755.946129] loop0: detected capacity change from 0 to 1099511627776 [12756.765487] SGI XFS with ACLs, security attributes, quota, no debug enabled [12756.785577] *** VALIDATE xfs *** [12756.799509] XFS (loop0): Mounting V5 Filesystem [12756.823906] XFS (loop0): Ending clean mount [12758.332788] loop1: detected capacity change from 0 to 18691697672192 [12869.389718] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [12884.272830] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [12924.208050] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12924.553599] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12924.624819] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12924.781573] Lustre: lustre-MDT0000: new disk, initializing [12924.996061] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12925.026362] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12932.019706] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12945.131839] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12945.198065] Lustre: 348704:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12945.237684] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12945.241133] Lustre: Skipped 1 previous similar message [12945.315733] Lustre: lustre-MDT0001: new disk, initializing [12945.516724] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12945.545068] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12951.134193] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12961.689655] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12968.839144] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12976.958224] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12977.237424] Lustre: lustre-OST0000: new disk, initializing [12977.240939] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12977.247122] Lustre: 350675:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12978.502308] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12978.514221] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12978.669996] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12984.297742] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12995.385568] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13001.007286] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [13001.176295] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [13008.069248] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [13008.446531] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [13014.500509] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13014.504110] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13014.508690] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13014.508698] Lustre: Skipped 4 previous similar messages [13014.520098] LustreError: Skipped 1 previous similar message [13014.551498] Lustre: Skipped 4 previous similar messages [13019.620460] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13019.635693] Lustre: Skipped 2 previous similar messages [13025.247294] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13025.721452] Lustre: server umount lustre-OST0000 complete [13032.416355] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13032.425977] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13032.439860] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13032.445189] Lustre: Skipped 2 previous similar messages [13037.927430] Lustre: server umount lustre-MDT0000 complete [13040.097781] LustreError: 348714:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13040.111635] LustreError: 348714:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [13041.981858] LustreError: 349499:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789788653 with bad export cookie 2665129459107737012 [13042.007282] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13042.014498] LustreError: 349499:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [13042.421794] Lustre: server umount lustre-MDT0001 complete [13052.145050] XFS (loop0): Unmounting Filesystem [13058.181379] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 23:31:07 (1789788667) [13066.221464] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13066.856432] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13066.861727] Lustre: Skipped 2 previous similar messages [13072.297053] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [13082.061549] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13087.253803] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [13096.342586] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13102.373480] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13111.469678] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13120.258486] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [13130.407925] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13143.013368] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13143.018841] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13143.025246] Lustre: Skipped 2 previous similar messages [13143.037477] Lustre: Skipped 3 previous similar messages [13146.568988] Lustre: server umount lustre-OST0000 complete [13157.057820] Lustre: server umount lustre-MDT0000 complete [13161.017296] LustreError: 353218:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789788772 with bad export cookie 2665129459107737894 [13161.022423] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13161.043870] LustreError: 353218:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [13161.481895] Lustre: server umount lustre-MDT0001 complete [13173.562472] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [13177.190188] Key type lgssc unregistered [13177.625713] LNet: 356867:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13177.634654] LNetError: 356867:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13177.650794] LNet: Removed LNI 192.168.202.111@tcp [13178.514161] Key type .llcrypt unregistered [13178.517340] Key type ._llcrypt unregistered [13214.816494] Key type ._llcrypt registered [13214.818410] Key type .llcrypt registered [13214.918227] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [13216.317235] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13216.610937] alg: No test for adler32 (adler32-zlib) [13217.814292] Lustre: Lustre: Build Version: 2.17.57_110_g5f5d390 [13218.134818] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [13219.791176] Key type lgssc registered [13221.281570] Lustre: Echo OBD driver; http://www.lustre.org/ [13231.904422] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13231.929468] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13233.634117] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13238.723555] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [13243.874182] LustreError: 358434:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13243.899208] LustreError: 358434:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [13248.993255] LustreError: 358435:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13250.037866] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13250.625912] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13258.004424] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [13267.424291] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13273.695669] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13281.157793] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13281.470691] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13288.452295] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [13297.068798] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13315.567574] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13315.577884] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13315.585437] Lustre: Skipped 1 previous similar message [13319.438540] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.11@tcp (stopping) [13320.672383] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13320.686368] Lustre: Skipped 1 previous similar message [13321.962492] Lustre: server umount lustre-MDT0001 complete [13324.498720] LustreError: 359339:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.11@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13329.643172] LustreError: 359339:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.11@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13329.673584] LustreError: 359339:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [13334.742044] LustreError: 358430:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.11@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13334.775994] LustreError: 358430:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [13336.672580] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13336.787163] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13336.790726] Lustre: Found index 1 for lustre-MDT0001, updating log [13336.842785] Lustre: 358424:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13337.246310] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13342.686627] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [13342.706515] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13346.312425] LustreError: 362058:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13346.322555] LustreError: 362058:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13346.336387] LustreError: 358426:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13346.419236] LustreError: 362061:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.111@tcp: cfg command failed: rc = -17 [13346.432155] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [13346.432155] [13649.897941] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13649.935387] Lustre: Skipped 1 previous similar message [13649.967156] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13649.971550] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13655.010517] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13655.030701] Lustre: Skipped 3 previous similar messages [13663.199161] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13663.478514] Lustre: server umount lustre-MDT0001 complete [13665.252655] LustreError: 358430:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13665.273956] LustreError: 358430:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [13673.678434] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13673.875374] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13673.879260] Lustre: Found index 1 for lustre-MDT0001, updating log [13673.916279] Lustre: 358423:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13674.195034] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13678.138426] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [13679.094448] LustreError: 369985:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13679.108965] LustreError: 369985:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13679.116357] LustreError: 358426:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13679.124901] LustreError: 358426:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13679.230496] LustreError: 369997:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13679.243309] LustreError: 369997:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13679.257158] LustreError: 369997:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.111@tcp: cfg command failed: rc = -17 [13679.268270] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.111@tcp [13679.268270] [13679.622264] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13985.009373] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.11@tcp (stopping) [13985.013295] Lustre: Skipped 4 previous similar messages [13986.787567] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13986.813542] Lustre: Skipped 1 previous similar message [13989.870525] Lustre: server umount lustre-MDT0001 complete [13990.099702] LustreError: 371919:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.11@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13990.134801] LustreError: 371919:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [14001.846252] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14001.962950] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [14001.976216] Lustre: Found index 1 for lustre-MDT0001, updating log [14002.053513] Lustre: 358423:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14002.425234] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [14007.785446] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [14007.797630] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [14011.390912] LustreError: 378407:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [14011.396373] LustreError: 378407:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [14011.402190] LustreError: 358426:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [14011.408099] LustreError: 358426:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [14011.480594] LustreError: 378410:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [14011.496472] LustreError: 378410:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.111@tcp: cfg command failed: rc = -114 [14011.502194] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [14011.502194] [14335.462169] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14335.462610] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14335.478556] Lustre: Skipped 2 previous similar messages [14335.479045] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14335.479053] Lustre: Skipped 2 previous similar messages [14338.955132] Lustre: server umount lustre-MDT0000 complete [14340.582371] LustreError: 358430:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14340.605203] LustreError: 358430:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [14342.338262] LustreError: 361643:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789789954 with bad export cookie 16982029896945914057 [14342.344811] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14342.359423] LustreError: 361643:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14345.699596] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14345.707705] Lustre: Skipped 1 previous similar message [14348.637952] Lustre: server umount lustre-MDT0001 complete [14352.411285] Lustre: server umount lustre-OST0000 complete [14361.278196] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [14372.124977] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [14410.724907] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 23:53:40 (1789790020) [14416.850353] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14417.155363] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14417.200323] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14417.402153] Lustre: lustre-MDT0000: new disk, initializing [14417.515180] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14417.548737] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14421.539308] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [14432.438826] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14432.612287] Lustre: 388886:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14432.763455] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14432.768265] Lustre: Skipped 1 previous similar message [14432.948787] Lustre: lustre-MDT0001: new disk, initializing [14433.116966] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14433.177655] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14433.196303] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14439.301397] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [14449.526588] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14456.315255] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14465.132449] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14465.493805] Lustre: lustre-OST0000: new disk, initializing [14465.496645] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14465.513486] Lustre: 390857:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14465.637310] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14472.030781] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [14475.910026] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14475.918983] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14476.096823] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14482.638223] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14491.623569] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14491.625634] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14491.652469] Lustre: Skipped 1 previous similar message [14491.688733] Lustre: Skipped 6 previous similar messages [14492.990740] Lustre: server umount lustre-MDT0000 complete [14496.746567] LustreError: 389808:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14496.798660] LustreError: 389808:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [14497.492420] LustreError: 388880:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789790109 with bad export cookie 16982029896945916010 [14497.501262] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14497.505624] LustreError: 388880:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [14498.063677] Lustre: server umount lustre-MDT0001 complete [14513.103461] Lustre: server umount lustre-OST0000 complete [14540.962930] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [14551.773065] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [14589.834704] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 14295 sec ========= 23:56:38 (1789790198) [14592.002676] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 23:56:41 (1789790201) === [14596.163225] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 23:56:45 (1789790205) === [14644.134361] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [14647.346493] Key type lgssc unregistered [14647.694329] LNet: 397224:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14647.698368] LNetError: 397224:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14647.721724] LNet: Removed LNI 192.168.202.111@tcp [14648.540186] Key type .llcrypt unregistered [14648.542580] Key type ._llcrypt unregistered