[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 505946641 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.003042] x2apic enabled [ 0.004007] Switched APIC routing to physical x2apic. [ 0.005013] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008020] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009010] pid_max: default: 32768 minimum: 301 [ 0.010131] LSM: Security Framework initializing [ 0.011084] Yama: becoming mindful. [ 0.012038] SELinux: Initializing. [ 0.013061] *** VALIDATE selinux *** [ 0.021166] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026423] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027157] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028103] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029109] *** VALIDATE tmpfs *** [ 0.031227] *** VALIDATE proc *** [ 0.032253] *** VALIDATE cgroup *** [ 0.033015] *** VALIDATE cgroup2 *** [ 0.034273] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035185] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037032] Spectre V2 : User space: Vulnerable [ 0.038011] Speculative Store Bypass: Vulnerable [ 0.041441] debug: unmapping init [mem 0xffffffff9c859000-0xffffffff9c860fff] [ 0.043170] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044720] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045028] ... version: 2 [ 0.046013] ... bit width: 48 [ 0.047013] ... generic registers: 4 [ 0.048013] ... value mask: 0000ffffffffffff [ 0.049016] ... max period: 00007fffffffffff [ 0.050013] ... fixed-purpose events: 3 [ 0.051012] ... event mask: 000000070000000f [ 0.052302] rcu: Hierarchical SRCU implementation. [ 0.054446] smp: Bringing up secondary CPUs ... [ 0.055627] x86: Booting SMP configuration: [ 0.056031] .... node #0, CPUs: #1 #2 #3 [ 0.059309] smp: Brought up 1 node, 4 CPUs [ 0.061012] smpboot: Max logical packages: 1 [ 0.062020] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.145606] node 0 deferred pages initialised in 81ms [ 0.148099] devtmpfs: initialized [ 0.149361] x86/mm: Memory block size: 128MB [ 0.152831] gcov: version magic: 0x41383552 [ 0.154297] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.158135] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.161367] pinctrl core: initialized pinctrl subsystem [ 0.163186] [ 0.163790] ************************************************************* [ 0.166010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.168009] ** ** [ 0.171012] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.173011] ** ** [ 0.175014] ** This means that this kernel is built to expose internal ** [ 0.178010] ** IOMMU data structures, which may compromise security on ** [ 0.180011] ** your system. ** [ 0.183010] ** ** [ 0.185011] ** If you see this message and you are not debugging the ** [ 0.187010] ** kernel, report this immediately to your vendor! ** [ 0.190051] ** ** [ 0.192010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.195011] ************************************************************* [ 0.200250] NET: Registered protocol family 16 [ 0.202458] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.205073] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.207054] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.211092] cpuidle: using governor menu [ 0.212690] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.214466] PCI: Using configuration type 1 for base access [ 0.217143] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.226189] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.229030] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.233055] cryptd: max_cpu_qlen set to 1000 [ 0.236240] ACPI: Added _OSI(Module Device) [ 0.238015] ACPI: Added _OSI(Processor Device) [ 0.240010] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.241009] ACPI: Added _OSI(Processor Aggregator Device) [ 0.246072] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.252553] ACPI: Interpreter enabled [ 0.254060] ACPI: PM: (supports S0 S3 S4 S5) [ 0.255009] ACPI: Using IOAPIC for interrupt routing [ 0.256068] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.259353] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.268576] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.271048] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.273019] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.276081] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.281697] acpiphp: Slot [2] registered [ 0.283111] acpiphp: Slot [5] registered [ 0.284103] acpiphp: Slot [6] registered [ 0.285091] acpiphp: Slot [7] registered [ 0.287100] acpiphp: Slot [8] registered [ 0.288131] acpiphp: Slot [9] registered [ 0.290192] acpiphp: Slot [10] registered [ 0.291106] acpiphp: Slot [3] registered [ 0.293157] acpiphp: Slot [4] registered [ 0.294097] acpiphp: Slot [11] registered [ 0.296163] acpiphp: Slot [12] registered [ 0.297085] acpiphp: Slot [13] registered [ 0.299113] acpiphp: Slot [14] registered [ 0.300095] acpiphp: Slot [15] registered [ 0.302094] acpiphp: Slot [16] registered [ 0.303087] acpiphp: Slot [17] registered [ 0.305093] acpiphp: Slot [18] registered [ 0.306089] acpiphp: Slot [19] registered [ 0.307086] acpiphp: Slot [20] registered [ 0.309097] acpiphp: Slot [21] registered [ 0.310075] acpiphp: Slot [22] registered [ 0.311079] acpiphp: Slot [23] registered [ 0.312090] acpiphp: Slot [24] registered [ 0.314176] acpiphp: Slot [25] registered [ 0.315111] acpiphp: Slot [26] registered [ 0.317117] acpiphp: Slot [27] registered [ 0.318098] acpiphp: Slot [28] registered [ 0.320093] acpiphp: Slot [29] registered [ 0.322190] acpiphp: Slot [30] registered [ 0.323101] acpiphp: Slot [31] registered [ 0.325078] PCI host bridge to bus 0000:00 [ 0.327020] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.329040] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.332024] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.335022] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.337024] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.340028] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.342193] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.345143] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.349449] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.360014] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.365059] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.367016] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.369010] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.371013] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.376053] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.378834] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.381033] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.383873] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.389014] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.401017] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.404936] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.410770] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.416016] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.423015] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.438014] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.449850] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.457016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.466017] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.483018] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.501558] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.509015] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.519015] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.538021] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.548083] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.564015] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.572016] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.585016] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.595651] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.602015] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.609017] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.623020] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.632881] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.638016] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.643014] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.656015] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.666320] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.668316] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.670311] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.672298] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.675230] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.680039] iommu: Default domain type: Passthrough [ 0.681391] SCSI subsystem initialized [ 0.682117] ACPI: bus type USB registered [ 0.684078] usbcore: registered new interface driver usbfs [ 0.685065] usbcore: registered new interface driver hub [ 0.687084] usbcore: registered new device driver usb [ 0.688143] pps_core: LinuxPPS API ver. 1 registered [ 0.690010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.693078] PTP clock support registered [ 0.695107] EDAC MC: Ver: 3.0.0 [ 0.696518] PCI: Using ACPI for IRQ routing [ 0.698490] NetLabel: Initializing [ 0.699009] NetLabel: domain hash size = 128 [ 0.701009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.702076] NetLabel: unlabeled traffic allowed by default [ 0.704140] vgaarb: loaded [ 0.706261] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.708013] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.716113] clocksource: Switched to clocksource kvm-clock [ 0.824221] VFS: Disk quotas dquot_6.6.0 [ 0.825787] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.828089] *** VALIDATE ramfs *** [ 0.828993] *** VALIDATE hugetlbfs *** [ 0.830191] pnp: PnP ACPI init [ 0.832225] pnp: PnP ACPI: found 6 devices [ 0.846492] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.849106] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.850791] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.852436] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.854267] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.856079] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.858192] NET: Registered protocol family 2 [ 0.860170] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.864113] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.867021] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.872816] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.876713] TCP: Hash tables configured (established 65536 bind 65536) [ 0.880154] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.884208] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.887928] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.891137] NET: Registered protocol family 1 [ 0.893610] RPC: Registered named UNIX socket transport module. [ 0.895343] RPC: Registered udp transport module. [ 0.896749] RPC: Registered tcp transport module. [ 0.898124] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.900086] NET: Registered protocol family 44 [ 0.901510] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.903258] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.904884] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.906648] PCI: CLS 0 bytes, default 64 [ 0.908110] Unpacking initramfs... [ 2.293527] debug: unmapping init [mem 0xffff8ab6fcc54000-0xffff8ab6fffbffff] [ 2.297410] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.299472] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.302509] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.794158] Initialise system trusted keyrings [ 2.795422] Key type blacklist registered [ 2.796902] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.805196] zbud: loaded [ 2.807859] *** VALIDATE nfs *** [ 2.808810] *** VALIDATE nfs4 *** [ 2.810890] pstore: using deflate compression [ 2.813865] Platform Keyring initialized [ 2.907369] NET: Registered protocol family 38 [ 2.909139] Key type asymmetric registered [ 2.911639] Asymmetric key parser 'x509' registered [ 2.914520] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.918100] io scheduler mq-deadline registered [ 2.919424] io scheduler kyber registered [ 2.921088] io scheduler bfq registered [ 2.922578] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.924781] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.927327] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.929372] ACPI: Power Button [PWRF] [ 2.934450] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.942494] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.957817] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.966256] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.980119] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.006378] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.034200] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.038168] Non-volatile memory driver v1.3 [ 3.039301] Linux agpgart interface v0.103 [ 3.066341] virtio_blk virtio1: [vda] 150400 512-byte logical blocks (77.0 MB/73.4 MiB) [ 3.068629] vda: detected capacity change from 0 to 77004800 [ 3.083849] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.086840] vdb: detected capacity change from 0 to 1073741824 [ 3.102824] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.106264] vdc: detected capacity change from 0 to 2621440000 [ 3.127522] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.130545] vdd: detected capacity change from 0 to 2621440000 [ 3.146666] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.149729] vde: detected capacity change from 0 to 4294967296 [ 3.167693] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.170130] vdf: detected capacity change from 0 to 4294967296 [ 3.178364] libphy: Fixed MDIO Bus: probed [ 3.184383] usbcore: registered new interface driver usbserial_generic [ 3.186328] usbserial: USB Serial support registered for generic [ 3.188356] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.191844] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.193300] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.195145] mousedev: PS/2 mouse device common for all mice [ 3.197329] rtc_cmos 00:05: RTC can wake from S4 [ 3.199480] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.201489] rtc_cmos 00:05: registered as rtc0 [ 3.204920] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.206485] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.207661] intel_pstate: CPU model not supported [ 3.212493] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.216468] hid: raw HID events driver (C) Jiri Kosina [ 3.217962] usbcore: registered new interface driver usbhid [ 3.219431] usbhid: USB HID core driver [ 3.220613] drop_monitor: Initializing network drop monitor service [ 3.222346] Initializing XFRM netlink socket [ 3.223965] NET: Registered protocol family 10 [ 3.226595] Segment Routing with IPv6 [ 3.227993] NET: Registered protocol family 17 [ 3.229877] mpls_gso: MPLS GSO support [ 3.237200] RAS: Correctable Errors collector initialized. [ 3.239889] AVX version of gcm_enc/dec engaged. [ 3.242619] AES CTR mode by8 optimization enabled [ 3.313455] sched_clock: Marking stable (3313434569, 0)->(4215968170, -902533601) [ 3.316188] registered taskstats version 1 [ 3.318164] Loading compiled-in X.509 certificates [ 3.320161] zswap: loaded using pool lzo/zbud [ 3.342337] Key type big_key registered [ 3.352373] Key type encrypted registered [ 3.353745] ima: No TPM chip found, activating TPM-bypass! [ 3.355283] ima: Allocated hash algorithm: sha1 [ 3.356420] ima: No architecture policies found [ 3.357655] evm: Initialising EVM extended attributes: [ 3.359453] evm: security.selinux [ 3.360301] evm: security.ima [ 3.361101] evm: security.capability [ 3.362045] evm: HMAC attrs: 0x1 [ 3.364219] rtc_cmos 00:05: setting system clock to 2026-10-01 23:06:49 UTC (1790896009) [ 3.370634] debug: unmapping init [mem 0xffffffff9d803000-0xffffffff9d9fffff] [ 3.373564] debug: unmapping init [mem 0xffffffff9c582000-0xffffffff9c858fff] [ 3.381053] Write protecting the kernel read-only data: 28672k [ 3.384042] debug: unmapping init [mem 0xffffffff9ac03000-0xffffffff9adfffff] [ 3.386225] debug: unmapping init [mem 0xffffffff9b514000-0xffffffff9b5fffff] [ 3.417238] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.425583] systemd[1]: Detected virtualization kvm. [ 3.427258] systemd[1]: Detected architecture x86-64. [ 3.428486] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.451907] systemd[1]: No hostname configured. [ 3.453761] systemd[1]: Set hostname to . [ 3.455565] random: systemd: uninitialized urandom read (16 bytes read) [ 3.458066] systemd[1]: Initializing machine ID from random generator. [ 3.497832] random: ln: uninitialized urandom read (6 bytes read) [ 3.574805] random: systemd: uninitialized urandom read (16 bytes read) [ 3.576346] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 3.580508] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 3.585649] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... Starting Setup Virtual Console... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Local File Systems. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Reached target Slices. Starting Create Volatile Files and Directories... Starting Apply Kernel Variables... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.131319] device-mapper: uevent: version 1.0.3 [ 4.133518] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [[ 4.766547] random: fast init done  OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 4.816277] scsi host0: ata_piix [ 4.840375] virtio_net virtio0 ens2: renamed from eth0 [ 4.846125] scsi host1: ata_piix [ 4.847754] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 4.850674] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 8.715404] dracut-initqueue[581]: RTNETLINK answers: File exists [ 9.871475] random: crng init done [ 9.873248] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.320535] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.489123] printk: systemd: 25 output lines suppressed due to ratelimiting [ 11.713440] SELinux: Disabled at runtime. [ 11.773560] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.780517] systemd[1]: Detected virtualization kvm. [ 11.781819] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.220661] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.223793] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.228265] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.234372] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.237883] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.244387] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.252428] systemd[1]: Listening on RPCbind Server Activation Socket. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Starting Remount Root and Kernel File Systems... [ OK ] Listening on udev Control Socket. Mounting POSIX Message Queue File System... Mounting Huge Pages File System... [ OK ] Listening on Process Core Dump Socket. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. Mounting Kernel Debug File System... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target rpc_pipefs.target. [ OK ] Reached target Local Encrypted Volumes. Starting Create list of required st…ce nodes for the current kernel... Activating swap /dev/disk/by-label/SWAP... Starting Apply Kernel Variables... [ OK ] Stopped target Initrd Root File System. [ OK ] Created slice system-getty.slice. [ OK ] Reached target Paths. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ 12.393618] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting udev Coldplug all Devices... [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 12.680556] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.013640] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.041806] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.152959] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.183366] EDAC sbridge: Ver: 1.1.2 [ 14.714537] Key type dns_resolver registered [ 15.017335] NFS: Registering the id_resolver key type [ 15.019212] Key type id_resolver registered [ 15.020851] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. [ OK ] Started Daily Cleanup of Temporary Directories. Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. Starting Login Service... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg131-server login: [ 50.834433] libcfs: loading out-of-tree module taints kernel. [ 50.866688] Key type ._llcrypt registered [ 50.869547] Key type .llcrypt registered [ 51.025461] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_hostid [ 69.504739] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 71.324949] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 71.339744] alg: No test for adler32 (adler32-zlib) [ 72.974613] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [ 74.138523] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [ 75.944286] Key type lgssc registered [ 78.033799] Lustre: Echo OBD driver; http://www.lustre.org/ [ 98.109036] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 137.008508] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 151.723809] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 151.810735] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 153.217744] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 153.241789] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 153.324910] Lustre: lustre-MDT0000: new disk, initializing [ 153.391780] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 153.405983] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 157.266646] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 171.215909] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 171.291190] Lustre: 6510:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 171.326193] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 171.332612] Lustre: Skipped 1 previous similar message [ 171.401280] Lustre: lustre-MDT0001: new disk, initializing [ 171.446643] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 171.466981] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 171.493839] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 175.285872] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 180.209200] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 189.572795] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 190.052282] Lustre: lustre-OST0000: new disk, initializing [ 190.061138] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 190.067714] Lustre: 8449:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 190.135122] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 196.115328] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 196.121378] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 196.203749] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 196.359474] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 210.631022] hrtimer: interrupt took 2600034 ns [ 210.848761] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 211.023856] Lustre: lustre-OST0001: new disk, initializing [ 211.028743] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 211.036608] Lustre: 9520:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 211.129269] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 216.801366] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 217.651567] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 217.663826] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 217.754955] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 229.806792] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 237.212352] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 243.288605] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing check_logdir /tmp/testlogs/ [ 248.983232] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing yml_node [ 254.416460] Lustre: DEBUG MARKER: Client: 2.17.58.63 [ 257.186210] Lustre: DEBUG MARKER: MDS: 2.17.58.63 [ 260.022063] Lustre: DEBUG MARKER: OSS: 2.17.58.63 [ 261.785633] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Oct 1 19:11:05 EDT 2026 [ 280.170910] Lustre: DEBUG MARKER: only running tests: 76-120 [ 282.092968] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 283.859584] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 304.611369] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 304.614686] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 304.617020] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 304.629428] Lustre: Skipped 2 previous similar messages [ 306.000867] Lustre: server umount lustre-MDT0000 complete [ 309.759374] LustreError: 6521:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 309.772565] LustreError: 10394:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790896315 with bad export cookie 4436768885408174878 [ 309.776618] LustreError: 6521:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 309.776954] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 309.809580] LustreError: 10394:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 314.850951] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 314.852375] LustreError: 6516:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 314.860320] Lustre: Skipped 1 previous similar message [ 314.860963] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 314.860968] Lustre: Skipped 3 previous similar messages [ 314.891675] LustreError: 6516:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 316.172247] Lustre: server umount lustre-MDT0001 complete [ 320.425899] Lustre: server umount lustre-OST0000 complete [ 324.387392] Lustre: server umount lustre-OST0001 complete [ 332.102719] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_hostid [ 340.144628] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 382.815626] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 383.080201] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 383.120152] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 383.217557] Lustre: lustre-MDT0000: new disk, initializing [ 383.332473] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 383.355190] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 388.189719] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 400.056706] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 400.208633] Lustre: 15124:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 400.262708] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 400.266255] Lustre: Skipped 1 previous similar message [ 400.341786] Lustre: lustre-MDT0001: new disk, initializing [ 400.423712] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 400.470926] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 400.484411] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 405.961976] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 420.461764] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 428.982192] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 436.240136] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 436.562850] Lustre: lustre-OST0000: new disk, initializing [ 436.567580] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 436.574459] Lustre: 17094:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 436.679832] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 438.365563] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 438.379299] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 438.549994] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 444.391943] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 454.947961] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 458.986807] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 459.166203] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 463.078795] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 463.289685] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 469.482470] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 469.489693] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 469.493450] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 469.498646] Lustre: Skipped 2 previous similar messages [ 469.513841] Lustre: Skipped 1 previous similar message [ 471.247835] Lustre: server umount lustre-OST0000 complete [ 477.153464] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 477.156660] LustreError: Skipped 1 previous similar message [ 477.163119] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 477.185078] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 481.314605] Lustre: server umount lustre-MDT0000 complete [ 484.838113] LustreError: 17177:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 484.866246] LustreError: 17177:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 485.218879] LustreError: 15117:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790896491 with bad export cookie 4436768885408176411 [ 485.230423] LustreError: 15117:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 485.235181] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 485.585045] Lustre: server umount lustre-MDT0001 complete [ 491.792454] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 19:14:55 (1790896495) [ 500.822318] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 501.233852] LustreError: 19360:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 501.305585] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 506.087598] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 506.344346] LustreError: 19361:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 514.860237] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 519.618492] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 526.823271] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 532.958584] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 540.453571] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 540.583422] LustreError: 21179:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 540.619967] LustreError: 21179:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 547.341111] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 556.233839] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 560.458817] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 571.362505] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 571.368072] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 571.375485] Lustre: Skipped 1 previous similar message [ 571.383722] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 571.388103] Lustre: Skipped 2 previous similar messages [ 574.361281] Lustre: server umount lustre-MDT0000 complete [ 576.999575] LustreError: 19361:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 577.011521] LustreError: 19361:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 578.089984] LustreError: 19342:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790896584 with bad export cookie 4436768885408177293 [ 578.105117] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 578.368733] Lustre: server umount lustre-MDT0001 complete [ 592.464323] Lustre: server umount lustre-OST0000 complete [ 615.446338] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 625.275881] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 625.623642] LustreError: 23743:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 625.684927] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 625.692375] Lustre: Skipped 2 previous similar messages [ 630.316466] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 639.372318] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 644.419088] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 648.340069] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 655.154255] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 662.373954] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 671.056078] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 671.258931] Lustre: lustre-OST0001: new disk, initializing [ 671.263482] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 671.269996] Lustre: 26060:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 672.395122] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 672.400703] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 672.538755] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 677.879663] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 688.009439] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 691.524919] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 706.534576] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 706.546696] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 706.557601] Lustre: Skipped 2 previous similar messages [ 706.561580] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 706.568690] Lustre: Skipped 2 previous similar messages [ 710.086989] Lustre: server umount lustre-MDT0000 complete [ 713.700190] LustreError: 24474:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 713.713096] LustreError: 24474:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 714.210492] LustreError: 26059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790896720 with bad export cookie 4436768885408178140 [ 714.211532] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 714.235050] LustreError: 26059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 714.758778] Lustre: server umount lustre-MDT0001 complete [ 728.186370] Lustre: server umount lustre-OST0000 complete [ 741.338125] Lustre: server umount lustre-OST0001 complete [ 759.179286] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 769.963282] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 770.395730] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 770.403982] Lustre: Skipped 3 previous similar messages [ 774.939134] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 784.705069] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 789.210487] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 792.537308] Lustre: 30228:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 799.731644] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 807.102973] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 816.530181] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 823.624420] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 832.230637] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 837.125496] Lustre: 32106:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 846.322965] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 846.332850] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 846.343268] Lustre: Skipped 3 previous similar messages [ 846.358558] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 846.368322] Lustre: Skipped 4 previous similar messages [ 850.718505] Lustre: server umount lustre-MDT0000 complete [ 852.967222] LustreError: 29080:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 852.976260] LustreError: 29080:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 9 previous similar messages [ 854.550472] LustreError: 29830:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790896860 with bad export cookie 4436768885408179435 [ 854.552192] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 854.564220] LustreError: 29830:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 854.972611] Lustre: server umount lustre-MDT0001 complete [ 869.765752] Lustre: server umount lustre-OST0000 complete [ 884.464126] Lustre: server umount lustre-OST0001 complete [ 893.376319] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 19:21:37 (1790896897) [ 931.438627] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 946.464966] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 951.464168] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 961.232733] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 966.631607] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 969.971052] Lustre: 36082:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 977.021369] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 984.358172] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 993.989497] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1001.851735] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 1011.187275] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1015.548390] Lustre: 37964:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1023.458460] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1023.467408] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1023.474912] Lustre: Skipped 3 previous similar messages [ 1023.478758] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1023.482993] Lustre: Skipped 4 previous similar messages [ 1027.390923] Lustre: server umount lustre-MDT0000 complete [ 1031.047297] LustreError: 37967:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790897037 with bad export cookie 4436768885408180534 [ 1031.054573] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1031.054670] LustreError: 37967:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1031.636642] Lustre: server umount lustre-MDT0001 complete [ 1046.779444] Lustre: server umount lustre-OST0000 complete [ 1049.698278] Lustre: 3644:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790897039/real 1790897039] req@ffff8ab77b29b100 x1877890648328960/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790897055 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1050.460891] Lustre: server umount lustre-OST0001 complete [ 1061.423831] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 19:24:25 (1790897065) [ 1078.548334] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 1090.127512] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1090.718636] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1090.724639] Lustre: Skipped 7 previous similar messages [ 1096.527500] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 1107.626413] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1113.958594] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 1117.676679] Lustre: 41257:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1124.589888] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1132.010921] LustreError: 41609:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1132.041108] LustreError: 41609:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 15 previous similar messages [ 1132.269410] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 1140.953199] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1147.462972] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 1155.976208] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1159.922320] Lustre: 43130:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1160.839873] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1160.844533] Lustre: Skipped 1 previous similar message [ 1164.262198] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1164.270052] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1164.287751] Lustre: Skipped 4 previous similar messages [ 1164.295605] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1164.299336] Lustre: Skipped 4 previous similar messages [ 1169.744052] Lustre: server umount lustre-MDT0000 complete [ 1173.898422] LustreError: 40088:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790897180 with bad export cookie 4436768885408181647 [ 1173.908164] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1173.911017] LustreError: 40088:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1188.320842] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1188.677603] Lustre: server umount lustre-MDT0001 complete [ 1197.372231] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1199.520716] LustreError: 43991:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 1199.538524] LustreError: 43991:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@ffff8ab77f7d8a80 x1877890648383488/t0(0) o250->MGC192.168.201.131@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1790897205 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1199.553843] LustreError: 43991:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 1199.586343] LustreError: 3640:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8ab77f7dbb80 x1877890648384640/t0(0) o250->MGC192.168.201.131@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1204.431106] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 1213.681595] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1215.348828] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1219.225418] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 1227.466247] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1229.207705] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1235.702775] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1237.598315] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1245.156204] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1249.881766] Lustre: server umount lustre-MDT0000 complete [ 1253.996066] LustreError: 40087:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790897260 with bad export cookie 4436768885408182739 [ 1253.996936] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1254.012039] LustreError: 40087:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1254.474720] Lustre: server umount lustre-MDT0001 complete [ 1270.015173] Lustre: server umount lustre-OST0000 complete [ 1272.800195] Lustre: 3641:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790897262/real 1790897262] req@ffff8ab766160a80 x1877890648420736/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790897278 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1275.100602] Lustre: server umount lustre-OST0001 complete [ 1288.879357] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 19:28:12 (1790897292) [ 1311.240975] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 1321.026385] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1325.459067] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 1334.014872] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1338.672320] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 1341.881358] Lustre: 48892:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1349.002989] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1356.057163] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 1365.933591] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1372.559932] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 1380.675933] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1385.980216] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1399.455896] Lustre: server umount lustre-MDT0000 complete [ 1400.803161] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1400.814957] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1400.823286] Lustre: Skipped 10 previous similar messages [ 1403.437526] LustreError: 47720:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790897409 with bad export cookie 4436768885408183565 [ 1403.447852] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1403.456370] LustreError: 47720:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1403.853753] Lustre: server umount lustre-MDT0001 complete [ 1418.758175] Lustre: server umount lustre-OST0000 complete [ 1433.476476] Lustre: server umount lustre-OST0001 complete [ 1443.884917] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 19:30:48 (1790897448) [ 1446.609386] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1448.470001] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 19:30:52 (1790897452) [ 1459.796155] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_hostid [ 1469.505571] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 1493.578807] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1493.937720] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1494.088958] Lustre: lustre-MDT0000: new disk, initializing [ 1494.205734] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1499.419477] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 1510.254348] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1518.182933] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1518.407460] Lustre: lustre-OST0000: new disk, initializing [ 1518.410414] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1518.413257] Lustre: Skipped 1 previous similar message [ 1518.417313] Lustre: 54969:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1520.419242] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1520.457923] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1520.502689] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1525.333618] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 1535.059258] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1537.680391] Lustre: DEBUG MARKER: create test files [ 1571.810866] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1571.819339] Lustre: Skipped 22 previous similar messages [ 1577.712801] Lustre: server umount lustre-OST0000 complete [ 1582.837318] Lustre: server umount lustre-MDT0000 complete [ 1597.632982] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [ 1600.987374] Key type lgssc unregistered [ 1601.454867] LNet: 56602:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1601.467992] LNetError: 56602:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1601.483268] LNet: Removed LNI 192.168.201.131@tcp [ 1602.755146] Key type .llcrypt unregistered [ 1602.760685] Key type ._llcrypt unregistered [ 1638.343812] Key type ._llcrypt registered [ 1638.346942] Key type .llcrypt registered [ 1638.473524] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 1640.272595] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1640.310932] alg: No test for adler32 (adler32-zlib) [ 1641.456855] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [ 1641.819813] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [ 1643.544236] Key type lgssc registered [ 1645.086665] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1654.937584] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1654.955679] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1656.545711] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1661.541555] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 1670.540823] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1679.019680] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1679.546455] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1681.606023] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1681.787740] Lustre: 59357:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1681.820921] Lustre: 59357:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1681.840571] Lustre: 59357:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1681.859438] Lustre: 59357:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1681.882640] Lustre: 59357:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1681.908408] Lustre: 59357:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1688.079491] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 1699.599418] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1705.935117] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1713.641315] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1727.973939] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1727.993549] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1728.017561] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1729.358620] Lustre: server umount lustre-OST0000 complete [ 1734.034577] Lustre: server umount lustre-MDT0000 complete [ 1747.263330] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [ 1749.780276] Key type lgssc unregistered [ 1750.128147] LNet: 60968:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1750.138339] LNetError: 60968:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1751.213381] LNet: Removed LNI 192.168.201.131@tcp [ 1752.202702] Key type .llcrypt unregistered [ 1752.207072] Key type ._llcrypt unregistered [ 1786.323571] Key type ._llcrypt registered [ 1786.326686] Key type .llcrypt registered [ 1786.417497] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 1787.755250] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1787.844303] alg: No test for adler32 (adler32-zlib) [ 1788.913618] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [ 1789.190911] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [ 1790.896237] Key type lgssc registered [ 1792.307761] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1802.917063] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1802.943692] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1804.586285] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1810.771331] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 1819.572401] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1825.891756] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1826.113901] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1829.178783] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1829.388360] Lustre: 63820:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1829.401658] Lustre: 63820:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1829.414054] Lustre: 63820:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1829.422310] Lustre: 63820:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1829.431491] Lustre: 63820:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1829.449529] Lustre: 63820:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1834.453685] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 1844.443134] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1849.870654] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1890.788237] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1890.797284] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1890.829216] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1894.190904] Lustre: server umount lustre-OST0000 complete [ 1898.153912] Lustre: server umount lustre-MDT0000 complete [ 1908.972072] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [ 1912.149899] Key type lgssc unregistered [ 1912.407898] LNet: 65402:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1912.414499] LNetError: 65402:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1912.441151] LNet: Removed LNI 192.168.201.131@tcp [ 1913.317693] Key type .llcrypt unregistered [ 1913.319592] Key type ._llcrypt unregistered [ 1931.741801] Key type ._llcrypt registered [ 1931.744267] Key type .llcrypt registered [ 1931.838796] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_hostid [ 1948.290816] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 1949.274277] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1949.528953] alg: No test for adler32 (adler32-zlib) [ 1950.637454] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [ 1950.956238] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [ 1952.664242] Key type lgssc registered [ 1953.864859] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1995.095990] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1995.111306] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1996.393447] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1996.455996] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1996.584924] Lustre: lustre-MDT0000: new disk, initializing [ 1996.671636] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1996.700940] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2000.254650] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 2012.784649] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2012.931769] Lustre: 68580:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2013.009724] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2013.016336] Lustre: Skipped 1 previous similar message [ 2013.106284] Lustre: lustre-MDT0001: new disk, initializing [ 2013.215848] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2013.275310] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2013.296348] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2019.717170] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 2030.469449] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2036.373148] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2043.959718] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2044.181978] Lustre: lustre-OST0000: new disk, initializing [ 2044.185323] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2044.194773] Lustre: 70545:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2044.320642] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2050.116468] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 2052.177740] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2052.200703] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2052.306921] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2061.494569] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2067.212649] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2067.421204] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2072.587401] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2072.784858] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2078.184469] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2078.190709] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2078.198709] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2078.203118] Lustre: Skipped 1 previous similar message [ 2081.047475] Lustre: server umount lustre-OST0000 complete [ 2088.424443] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2088.429841] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2088.457757] Lustre: Skipped 1 previous similar message [ 2088.488036] Lustre: Skipped 2 previous similar messages [ 2093.538640] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2093.552168] Lustre: Skipped 1 previous similar message [ 2098.658144] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2098.672402] Lustre: Skipped 1 previous similar message [ 2100.192141] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2100.418447] Lustre: server umount lustre-MDT0000 complete [ 2104.505722] LustreError: 68570:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790898110 with bad export cookie 14782039246722325096 [ 2104.522422] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2104.522808] LustreError: 68570:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2104.882311] Lustre: server umount lustre-MDT0001 complete [ 2115.831677] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 19:41:59 (1790898119) [ 2131.670383] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 2181.486390] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_hostid [ 2193.162407] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 2232.585299] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2233.044370] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2233.128189] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2233.248747] Lustre: lustre-MDT0000: new disk, initializing [ 2233.332235] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2233.358735] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2239.242895] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 2255.451146] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2255.725428] Lustre: 76588:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2255.818196] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2255.822504] Lustre: Skipped 1 previous similar message [ 2255.961092] Lustre: lustre-MDT0001: new disk, initializing [ 2256.154278] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2256.278721] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2256.291076] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2264.130330] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 2275.486356] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2282.426669] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2290.275950] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2290.637375] Lustre: lustre-OST0000: new disk, initializing [ 2290.647780] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2290.655895] Lustre: 78559:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2290.757334] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2296.985263] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 2297.398584] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2297.414995] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2297.512048] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2306.633956] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2312.160618] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2312.387741] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2317.129153] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2317.329467] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2323.432996] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2323.444663] LustreError: Skipped 1 previous similar message [ 2323.446689] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2323.451110] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2323.479743] Lustre: Skipped 1 previous similar message [ 2325.345158] Lustre: server umount lustre-OST0000 complete [ 2333.176938] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2333.179721] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2333.193977] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2333.200739] Lustre: Skipped 1 previous similar message [ 2336.049180] Lustre: server umount lustre-MDT0000 complete [ 2338.787533] LustreError: 77582:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2338.800758] LustreError: 77582:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [ 2339.938667] LustreError: 76583:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790898346 with bad export cookie 14782039246722325978 [ 2339.951331] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2339.952969] LustreError: 76583:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2340.401726] Lustre: server umount lustre-MDT0001 complete [ 2350.581661] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 19:45:54 (1790898354) [ 2357.773736] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2358.236171] LustreError: 80971:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2358.324258] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2363.372639] LustreError: 80972:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2363.477983] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 2368.482662] LustreError: 80971:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2372.323492] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2377.251278] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 2385.963375] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2392.020184] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2398.998850] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2399.293245] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2399.296242] Lustre: Skipped 1 previous similar message [ 2406.797189] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 2414.096865] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2417.658667] Lustre: 83642:0:(genops.c:1773:obd_export_evict_by_uuid()) MGS: evicting fdb6c29f-f989-4d1b-a663-b72c74a399bf at adminstrative request [ 2417.714707] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2417.738944] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2427.881126] Lustre: MGS: Client fdb6c29f-f989-4d1b-a663-b72c74a399bf (at 0@lo) reconnecting [ 2427.892903] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xcd245a6054c46145 to 0xcd245a6054c463ec [ 2427.899313] Lustre: MGC192.168.201.131@tcp: Connection restored to 0@lo (at 0@lo) [ 2457.483794] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2457.641551] Lustre: lustre-OST0001: new disk, initializing [ 2457.645121] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2457.650978] Lustre: 84039:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2465.035720] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2465.048465] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2465.252779] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2466.297597] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 2480.816847] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2485.730879] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2485.759346] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2485.772629] Lustre: Skipped 1 previous similar message [ 2485.788521] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2485.802042] Lustre: Skipped 2 previous similar messages [ 2490.700587] Lustre: server umount lustre-OST0001 complete [ 2490.867204] LustreError: 82794:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2495.458942] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2495.467293] LustreError: Skipped 1 previous similar message [ 2495.472432] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2495.495948] Lustre: Skipped 1 previous similar message [ 2501.107135] LustreError: 82793:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2501.129263] LustreError: 82793:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [ 2501.379146] Lustre: server umount lustre-OST0000 complete [ 2512.328804] Lustre: server umount lustre-MDT0000 complete [ 2516.707799] LustreError: 82815:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790898522 with bad export cookie 14782039246722327532 [ 2516.712172] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2516.722932] LustreError: 82815:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2517.336085] Lustre: server umount lustre-MDT0001 complete [ 2527.360650] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 19:48:51 (1790898531) [ 2529.099114] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2531.040829] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 19:48:55 (1790898535) [ 2533.003797] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2535.269486] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 19:48:59 (1790898539) [ 2537.010425] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2539.532947] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 19:49:03 (1790898543) [ 2541.397965] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2543.318840] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 19:49:07 (1790898547) [ 2545.049624] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2552.262120] loop: module loaded [ 2552.276426] loop0: detected capacity change from 0 to 4294967296 [ 2552.325220] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2554.276418] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2558.083946] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2561.765694] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2590.863706] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2590.973514] LustreError: 88164:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2590.995995] LustreError: 88164:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2591.007351] LustreError: 88164:0:(tgt_mount.c:2581:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2591.033974] LustreError: 88164:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2617.630512] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_hostid [ 2631.617871] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 2677.351231] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2677.647985] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2677.689100] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2677.816085] Lustre: lustre-MDT0000: new disk, initializing [ 2677.944604] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2677.947843] Lustre: Skipped 1 previous similar message [ 2677.986252] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2683.366219] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 2697.450516] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2697.704263] Lustre: 91032:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2698.171368] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2703.447904] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 2714.903416] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2722.709517] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2730.588831] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2730.948198] Lustre: 93001:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2737.318407] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2737.485127] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2738.674523] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 2751.807892] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2758.403728] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2758.580909] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2764.539299] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2764.766544] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2768.365518] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2768.369971] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2768.378087] LustreError: Skipped 1 previous similar message [ 2768.381972] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2768.390535] Lustre: Skipped 4 previous similar messages [ 2768.404587] Lustre: Skipped 8 previous similar messages [ 2781.152721] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2781.337502] Lustre: server umount lustre-OST0000 complete [ 2793.816860] Lustre: server umount lustre-MDT0000 complete [ 2793.963580] LustreError: 91843:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2793.986707] LustreError: 91843:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 2798.346585] LustreError: 91024:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790898804 with bad export cookie 14782039246722328148 [ 2798.351340] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2798.360886] LustreError: 91024:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2798.618656] Lustre: server umount lustre-MDT0001 complete [ 2808.974624] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 19:53:32 (1790898812) [ 2817.596821] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2818.427356] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 2818.434210] Lustre: Skipped 2 previous similar messages [ 2823.717497] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 2828.774453] LustreError: 95419:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2828.798501] LustreError: 95419:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 2833.767196] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2839.276906] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 2848.474602] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2856.672354] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2866.337105] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2874.705176] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 2884.950607] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2890.033254] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2890.267626] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2896.136472] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2896.329952] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2903.819396] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2904.027380] Lustre: lustre-OST0001: new disk, initializing [ 2904.031201] Lustre: Skipped 2 previous similar messages [ 2904.044879] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2904.049289] Lustre: Skipped 3 previous similar messages [ 2904.056995] Lustre: 98789:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2911.107719] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 2913.923329] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2913.936586] Lustre: Skipped 2 previous similar messages [ 2913.945245] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2914.044968] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2922.483473] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2928.742309] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2929.005536] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2933.956627] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2934.223656] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2957.377407] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2987.298768] Lustre: Failing over lustre-MDT0000 [ 2987.489808] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2987.506661] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2987.526070] Lustre: Skipped 2 previous similar messages [ 2987.536681] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2987.540340] Lustre: Skipped 7 previous similar messages [ 2987.796919] Lustre: server umount lustre-MDT0000 complete [ 2988.922343] LustreError: 97108:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.31@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2988.945209] LustreError: 97108:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 3007.456164] Lustre: 66791:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790898997/real 1790898997] req@0000000059df7511 x1877892616452480/t0(0) o400->MGC192.168.201.131@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1790899013 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3007.477325] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3016.470169] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3016.472566] LDISKFS-fs (dm-0): recovery complete [ 3016.491830] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3017.826595] LustreError: 101608:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 3017.836953] LustreError: 101608:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@00000000524cc14c x1877892616458112/t0(0) o250->MGC192.168.201.131@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1790899023 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3017.877368] LustreError: 101608:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 3018.723425] LustreError: 66787:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@000000005069799d x1877892616461440/t0(0) o250->MGC192.168.201.131@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3019.420222] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3020.653150] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3020.662275] Lustre: 95413:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3024.369639] Lustre: 95418:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3024.377626] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3024.399508] Lustre: 95418:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 1 previous similar message [ 3024.522119] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3024.856182] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3025.187951] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3025.200253] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 3025.359315] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 3025.538666] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3025.552871] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 3025.569866] Lustre: 101639:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3025.583184] Lustre: 101639:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 7 previous similar messages [ 3026.233981] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3026.249803] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 2 previous similar messages [ 3026.576137] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3026.584608] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 2 previous similar messages [ 3027.578137] Lustre: 101639:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3027.601417] Lustre: 101639:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 5 previous similar messages [ 3028.276048] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3028.288633] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 5 previous similar messages [ 3028.616777] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3028.632601] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 5 previous similar messages [ 3031.879880] Lustre: 101639:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3031.890183] Lustre: 101639:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 12 previous similar messages [ 3032.543441] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3032.566734] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 3032.897336] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3032.906770] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 3040.083986] Lustre: 101639:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3040.100163] Lustre: 101639:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 24 previous similar messages [ 3040.745356] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3040.761407] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 24 previous similar messages [ 3041.096606] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3041.116622] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 24 previous similar messages [ 3056.192992] Lustre: 101639:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3056.205192] Lustre: 101639:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 49 previous similar messages [ 3056.845268] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3056.865147] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 49 previous similar messages [ 3057.184204] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3057.197065] LustreError: 101639:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 49 previous similar messages [ 3080.168199] Lustre: 101639:0:(ldlm_lib.c:2428:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3080.187733] Lustre: 101639:0:(ldlm_lib.c:2438:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3080.203119] Lustre: 101639:0:(genops.c:1600:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 9a5766a2-7fe2-4e37-8735-1a4bd4e41d83@192.168.201.31@tcp [ 3080.238788] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3080.256772] Lustre: 101639:0:(ldlm_lib.c:1948:abort_req_replay_queue()) @@@ aborted: req@0000000077c9767c x1877892598402048/t0(8589934769) o101->9a5766a2-7fe2-4e37-8735-1a4bd4e41d83@192.168.201.31@tcp:592/0 lens 592/0 e 0 to 0 dl 1790899097 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3080.315760] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3080.324844] Lustre: lustre-MDT0000: Denying connection for new client 9a5766a2-7fe2-4e37-8735-1a4bd4e41d83 (at 192.168.201.31@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3080.355497] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3080.384188] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3080.387090] Lustre: Skipped 2 previous similar messages [ 3080.389878] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3080.411395] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 3080.411614] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 3099.620780] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3099.633516] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3099.645719] Lustre: Skipped 3 previous similar messages [ 3111.392398] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3111.577602] Lustre: server umount lustre-OST0000 complete [ 3119.091665] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 3119.105555] Lustre: Skipped 7 previous similar messages [ 3121.655197] LustreError: 102022:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3121.671125] LustreError: 102022:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 47 previous similar messages [ 3123.924428] Lustre: server umount lustre-OST0001 complete [ 3134.532656] Lustre: server umount lustre-MDT0000 complete [ 3138.925236] LustreError: 96153:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790899145 with bad export cookie 14782039246722379360 [ 3138.927374] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3138.939135] LustreError: 96153:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3139.691765] Lustre: server umount lustre-MDT0001 complete [ 3149.834674] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 19:59:13 (1790899153) [ 3159.410391] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3159.422785] Lustre: *** cfs_fail_loc=197, val=0*** [ 3175.204481] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3180.320747] LustreError: 103623:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.131@tcp: failed processing log, type 4: rc = -110 [ 3185.483733] Lustre: *** cfs_fail_loc=197, val=0*** [ 3185.490778] Lustre: Skipped 65 previous similar messages [ 3206.112361] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3206.121774] Lustre: Skipped 4 previous similar messages [ 3212.501668] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 3221.046283] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3223.717279] Lustre: server umount lustre-OST0000 complete [ 3233.403257] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 20:00:37 (1790899237) [ 3257.523066] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3264.243207] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 3278.374307] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3282.711134] LustreError: 105288:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3282.729321] LustreError: 105288:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3282.736471] LustreError: 105288:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 24, retries 0, failed: rc = -5 [ 3283.016022] Lustre: server umount lustre-MDT0000 complete [ 3321.427975] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_hostid [ 3334.428365] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 3377.917436] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3378.267515] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3378.355764] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3378.432478] Lustre: lustre-MDT0000: new disk, initializing [ 3378.580606] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3383.629436] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 3397.621873] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3397.751861] Lustre: 109466:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3397.971882] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3402.913157] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 3413.471337] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3420.068313] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3428.292104] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3428.742710] Lustre: 111433:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3435.370048] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3437.662358] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 3449.344747] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3456.144817] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3456.443688] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3463.037096] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3463.279554] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3466.209547] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3466.213249] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3466.221049] LustreError: Skipped 3 previous similar messages [ 3466.223914] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3466.223924] Lustre: Skipped 4 previous similar messages [ 3466.260976] Lustre: Skipped 6 previous similar messages [ 3472.544479] Lustre: server umount lustre-OST0000 complete [ 3484.756593] Lustre: server umount lustre-MDT0000 complete [ 3486.694346] LustreError: 109472:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3486.728937] LustreError: 109472:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 8 previous similar messages [ 3489.104413] LustreError: 109457:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790899495 with bad export cookie 14782039246722389937 [ 3489.109571] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3489.116088] LustreError: 109457:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 3489.542606] Lustre: server umount lustre-MDT0001 complete [ 3501.306037] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 20:05:04 (1790899504) [ 3512.869426] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [ 3515.741622] Key type lgssc unregistered [ 3516.044764] LNet: 113921:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3516.049571] LNetError: 113921:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3516.070614] LNet: Removed LNI 192.168.201.131@tcp [ 3516.757938] Key type .llcrypt unregistered [ 3516.759787] Key type ._llcrypt unregistered [ 3535.313862] Key type ._llcrypt registered [ 3535.316264] Key type .llcrypt registered [ 3535.390415] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_hostid [ 3552.350370] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 3553.473576] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3553.648028] alg: No test for adler32 (adler32-zlib) [ 3554.749489] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [ 3554.973215] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [ 3556.658160] Key type lgssc registered [ 3557.929735] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3602.787310] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3602.813629] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3604.295972] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3604.348717] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3604.496565] Lustre: lustre-MDT0000: new disk, initializing [ 3604.678557] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3604.723299] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3610.045332] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 3622.968140] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3623.397505] Lustre: lustre-OST0000: new disk, initializing [ 3623.404289] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3623.408788] Lustre: Skipped 1 previous similar message [ 3623.417913] Lustre: 118114:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3623.545049] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3631.267146] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 3632.251934] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3632.256632] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3632.307122] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3643.147880] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3643.328749] Lustre: lustre-OST0001: new disk, initializing [ 3643.331883] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3643.335501] Lustre: 118979:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3643.390465] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3650.379038] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 3652.266682] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3652.274743] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3652.295153] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3662.316991] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3662.367808] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3665.906728] Lustre: server umount lustre-MDT0000 complete [ 3672.955896] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3678.029189] LustreError: 117299:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790899684 with bad export cookie 8354452873507377611 [ 3678.032707] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3678.037593] LustreError: 117299:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3678.085887] Lustre: server umount lustre-OST0000 complete [ 3682.826765] Lustre: server umount lustre-OST0001 complete [ 3693.606990] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 20:08:17 (1790899697) [ 3720.161435] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 20:08:44 (1790899724) [ 3731.489839] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 3759.749949] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 20:09:22 (1790899762) [ 3804.557873] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_hostid [ 3815.703567] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 3858.703698] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3858.920579] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3858.957703] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3859.061592] Lustre: lustre-MDT0000: new disk, initializing [ 3859.171256] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3859.200688] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3864.775575] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 3877.619908] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3877.799996] Lustre: 126739:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3877.877179] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3877.883111] Lustre: Skipped 1 previous similar message [ 3877.955749] Lustre: lustre-MDT0001: new disk, initializing [ 3878.045670] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3878.087586] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3878.104436] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3883.565047] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 3896.317629] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3903.744877] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3911.573386] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3912.032310] Lustre: lustre-OST0000: new disk, initializing [ 3912.038291] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3912.045681] Lustre: 128714:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3912.240416] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3919.428725] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3919.467457] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3919.594872] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3920.445832] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 3934.314581] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3941.467907] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3941.679860] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3947.963383] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3948.283426] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3955.683313] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3955.698888] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3955.711671] Lustre: Skipped 1 previous similar message [ 3955.723427] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3955.730086] Lustre: Skipped 2 previous similar messages [ 3957.056070] Lustre: server umount lustre-OST0000 complete [ 3965.410079] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3965.417232] LustreError: Skipped 1 previous similar message [ 3965.421068] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3965.435296] Lustre: Skipped 1 previous similar message [ 3965.440800] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3968.554939] Lustre: server umount lustre-MDT0000 complete [ 3972.750078] LustreError: 126733:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790899978 with bad export cookie 8354452873507378836 [ 3972.756850] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3972.758239] LustreError: 126733:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3973.057820] Lustre: server umount lustre-MDT0001 complete [ 3983.042596] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 20:13:07 (1790899987) [ 3990.937467] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3991.621951] LustreError: 131129:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3991.669608] LustreError: 131129:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 3991.841313] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3997.162116] LustreError: 131130:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3997.264690] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 4001.252590] LustreError: 131129:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4006.376422] LustreError: 131130:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4007.412538] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4012.362815] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 4020.435534] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4026.880811] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4034.571149] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4034.848247] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4034.854426] Lustre: Skipped 1 previous similar message [ 4040.991797] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 4049.557370] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4057.901829] Lustre: *** cfs_fail_loc=159, val=0*** [ 4057.915881] Lustre: Skipped 2 previous similar messages [ 4073.399675] Lustre: lustre-MDT0000: Client 1b0ffa2b-6154-4ec8-8e40-24c2facab8d8 (at 192.168.201.31@tcp) reconnecting [ 4075.433924] Lustre: *** cfs_fail_loc=159, val=0*** [ 4075.442868] Lustre: Skipped 3 previous similar messages [ 4091.745866] Lustre: lustre-MDT0000: Client 1b0ffa2b-6154-4ec8-8e40-24c2facab8d8 (at 192.168.201.31@tcp) reconnecting [ 4098.532026] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4098.535244] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4098.542186] Lustre: Skipped 1 previous similar message [ 4098.543683] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4098.543688] Lustre: Skipped 1 previous similar message [ 4103.651032] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4103.668340] Lustre: Skipped 2 previous similar messages [ 4104.673288] Lustre: server umount lustre-OST0000 complete [ 4108.773245] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4108.790047] Lustre: Skipped 2 previous similar messages [ 4113.899697] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4113.909751] Lustre: Skipped 3 previous similar messages [ 4115.106436] Lustre: server umount lustre-MDT0000 complete [ 4119.015428] LustreError: 131129:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4119.035493] LustreError: 131129:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4119.181884] LustreError: 133849:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790900125 with bad export cookie 8354452873507379718 [ 4119.184174] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4119.197521] LustreError: 133849:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4119.524370] Lustre: server umount lustre-MDT0001 complete [ 4130.554334] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [ 4134.335709] Key type lgssc unregistered [ 4134.735764] LNet: 134903:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4134.745473] LNetError: 134903:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4134.769067] LNet: Removed LNI 192.168.201.131@tcp [ 4136.005673] Key type .llcrypt unregistered [ 4136.008263] Key type ._llcrypt unregistered [ 4165.707268] Key type ._llcrypt registered [ 4165.711920] Key type .llcrypt registered [ 4165.845743] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 4166.704089] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4166.781533] alg: No test for adler32 (adler32-zlib) [ 4168.049081] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [ 4168.260969] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [ 4169.920213] Key type lgssc registered [ 4171.141049] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4179.349584] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4179.387108] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4180.939986] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4185.754070] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 4191.204951] LustreError: 136346:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4191.237702] LustreError: 136346:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4195.452034] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4195.899933] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4201.439661] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 4210.737255] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4218.143281] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4227.385648] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4228.059043] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4238.729897] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 4251.699038] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4253.737687] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4264.975615] Lustre: *** cfs_fail_loc=159, val=0*** [ 4281.243338] Lustre: lustre-MDT0000: Client 849eddd6-d752-4e87-ba21-a31e442eec1e (at 192.168.201.31@tcp) reconnecting [ 4283.662729] Lustre: *** cfs_fail_loc=159, val=0*** [ 4300.171692] Lustre: lustre-MDT0001: Client 849eddd6-d752-4e87-ba21-a31e442eec1e (at 192.168.201.31@tcp) reconnecting [ 4301.985252] Lustre: *** cfs_fail_loc=159, val=0*** [ 4301.991531] Lustre: Skipped 4 previous similar messages [ 4317.640960] Lustre: lustre-MDT0001: Client 849eddd6-d752-4e87-ba21-a31e442eec1e (at 192.168.201.31@tcp) reconnecting [ 4324.612560] Lustre: 139383:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4328.007096] Lustre: *** cfs_fail_loc=159, val=0*** [ 4328.017838] Lustre: Skipped 2 previous similar messages [ 4343.731044] Lustre: lustre-MDT0000: Client c7329d7a-6b55-4664-a202-80b396b2ad43 (at 192.168.201.31@tcp) reconnecting [ 4346.143106] Lustre: *** cfs_fail_loc=159, val=0*** [ 4346.152231] Lustre: Skipped 14 previous similar messages [ 4361.637646] Lustre: lustre-MDT0000: Client c7329d7a-6b55-4664-a202-80b396b2ad43 (at 192.168.201.31@tcp) reconnecting [ 4363.883780] Lustre: 139674:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4383.714082] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4383.723919] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4383.752112] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4384.738083] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4384.746387] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4384.754796] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4389.786886] Lustre: server umount lustre-OST0000 complete [ 4395.030753] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4395.062629] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4395.081182] Lustre: Skipped 2 previous similar messages [ 4400.097764] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4400.100649] Lustre: Skipped 1 previous similar message [ 4401.210179] Lustre: server umount lustre-MDT0000 complete [ 4405.218613] LustreError: 136346:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4405.243358] LustreError: 136346:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4406.614774] LustreError: 136326:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790900412 with bad export cookie 12264685249153860878 [ 4406.619252] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4406.632388] LustreError: 136326:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4407.553944] Lustre: server umount lustre-MDT0001 complete [ 4421.273775] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [ 4424.298143] Key type lgssc unregistered [ 4424.666049] LNet: 140754:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4424.679781] LNetError: 140754:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4424.709579] LNet: Removed LNI 192.168.201.131@tcp [ 4425.918412] Key type .llcrypt unregistered [ 4425.920668] Key type ._llcrypt unregistered [ 4456.356783] Key type ._llcrypt registered [ 4456.360581] Key type .llcrypt registered [ 4456.469238] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 4457.437391] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4457.470558] alg: No test for adler32 (adler32-zlib) [ 4458.577364] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [ 4458.828064] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [ 4460.576678] Key type lgssc registered [ 4462.331438] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4472.130812] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4472.161766] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4473.678332] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4478.854406] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 4483.058051] LustreError: 142185:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4483.085557] LustreError: 142185:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4488.162825] LustreError: 142186:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4489.268785] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4489.744990] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4495.259559] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 4505.292101] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4512.920215] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4520.881422] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4521.447737] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4527.917075] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 4534.783671] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4537.665370] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4542.964548] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4548.163990] Lustre: 144767:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4551.368113] Lustre: 144815:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4552.918524] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4553.853885] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4558.905625] Lustre: 144961:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4559.800581] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4560.957563] Lustre: 145059:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4568.580371] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4568.586766] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4568.591592] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4568.593390] Lustre: Skipped 1 previous similar message [ 4568.616839] Lustre: Skipped 1 previous similar message [ 4571.943747] Lustre: server umount lustre-OST0000 complete [ 4576.741452] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4576.750226] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4576.765869] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4578.785227] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4578.786645] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4583.180510] Lustre: server umount lustre-MDT0000 complete [ 4583.909562] LustreError: 142186:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4583.930172] LustreError: 142186:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4588.014634] LustreError: 142185:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4588.038937] LustreError: 142185:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4588.802327] LustreError: 142165:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790900594 with bad export cookie 12237401837950747370 [ 4588.827113] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4589.429486] Lustre: server umount lustre-MDT0001 complete [ 4604.769225] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [ 4608.059561] Key type lgssc unregistered [ 4608.340700] LNet: 146123:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4608.358419] LNetError: 146123:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4609.382906] LNet: Removed LNI 192.168.201.131@tcp [ 4610.257601] Key type .llcrypt unregistered [ 4610.262227] Key type ._llcrypt unregistered [ 4639.864942] Key type ._llcrypt registered [ 4639.867913] Key type .llcrypt registered [ 4639.963573] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 4641.445086] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4641.500410] alg: No test for adler32 (adler32-zlib) [ 4642.654112] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [ 4642.954653] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [ 4644.704397] Key type lgssc registered [ 4646.000541] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4654.060580] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4654.083141] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4655.523289] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4660.064447] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 4665.829750] LustreError: 147555:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4665.841056] LustreError: 147555:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4668.855316] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4669.313746] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4674.112809] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 4683.526379] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4689.641147] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4697.694199] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4698.036040] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4700.090488] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4706.624915] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 4714.439632] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4716.101549] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4722.392516] Lustre: *** cfs_fail_loc=159, val=0*** [ 4738.921963] Lustre: lustre-MDT0000: Client a5678ba8-e4af-4e19-9a73-b0e6a38d80d0 (at 192.168.201.31@tcp) reconnecting [ 4756.452779] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4756.464493] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4756.489793] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4760.546288] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4760.549806] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4760.562309] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4761.892588] Lustre: server umount lustre-OST0000 complete [ 4765.678544] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4765.689358] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4765.707018] Lustre: Skipped 1 previous similar message [ 4765.723509] Lustre: Skipped 2 previous similar messages [ 4770.786697] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4770.791589] Lustre: Skipped 1 previous similar message [ 4771.823114] Lustre: server umount lustre-MDT0000 complete [ 4775.796180] LustreError: 147535:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790900781 with bad export cookie 16436287014834964177 [ 4775.797261] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4775.802533] LustreError: 147535:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4776.140049] Lustre: server umount lustre-MDT0001 complete [ 4788.021763] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [ 4790.700569] Key type lgssc unregistered [ 4791.001670] LNet: 151221:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4791.008861] LNetError: 151221:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4791.028533] LNet: Removed LNI 192.168.201.131@tcp [ 4791.788112] Key type .llcrypt unregistered [ 4791.790412] Key type ._llcrypt unregistered [ 4819.578172] Key type ._llcrypt registered [ 4819.580128] Key type .llcrypt registered [ 4819.675150] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 4820.753637] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4820.921035] alg: No test for adler32 (adler32-zlib) [ 4822.022828] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [ 4822.211517] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [ 4823.872231] Key type lgssc registered [ 4825.078744] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4832.155692] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4832.188475] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4833.719406] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4839.341606] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 4842.985705] LustreError: 152654:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4843.003321] LustreError: 152654:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4848.075564] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4848.100206] LustreError: 152655:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4848.495656] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4853.006803] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 4861.429444] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4867.715700] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4875.324865] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4875.713522] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4878.787143] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4882.716317] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 4891.514684] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4893.113285] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4912.187326] Lustre: lustre-MDT0001: evicting d5de6a92-e08d-4f48-8e30-f93fcd4f58b8 (at 192.168.201.31@tcp) by administrative request [ 4912.203750] Lustre: Skipped 1 previous similar message [ 4944.357721] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4944.361340] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4944.374536] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4944.395627] Lustre: Skipped 1 previous similar message [ 4947.922868] Lustre: server umount lustre-OST0000 complete [ 4954.597440] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4954.599961] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4954.609893] Lustre: Skipped 1 previous similar message [ 4954.628082] Lustre: Skipped 2 previous similar messages [ 4959.714530] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4959.740638] Lustre: Skipped 1 previous similar message [ 4964.840092] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4964.843195] Lustre: Skipped 1 previous similar message [ 4966.368152] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4966.545441] Lustre: server umount lustre-MDT0000 complete [ 4969.955580] LustreError: 152655:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4969.968076] LustreError: 152655:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4970.503630] LustreError: 152635:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790900976 with bad export cookie 6185645625369848746 [ 4970.505354] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4970.535673] LustreError: 152635:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4970.772748] Lustre: server umount lustre-MDT0001 complete [ 4983.756364] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [ 4986.989485] Key type lgssc unregistered [ 4987.347963] LNet: 156727:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4987.360291] LNetError: 156727:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4988.396845] LNet: Removed LNI 192.168.201.131@tcp [ 4989.654643] Key type .llcrypt unregistered [ 4989.657208] Key type ._llcrypt unregistered [ 5066.489466] Key type ._llcrypt registered [ 5066.490747] Key type .llcrypt registered [ 5066.646309] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 5068.183114] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5068.237074] alg: No test for adler32 (adler32-zlib) [ 5069.372838] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [ 5069.600335] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [ 5071.376401] Key type lgssc registered [ 5072.727654] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5081.833538] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5081.885715] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5083.712805] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5088.490895] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 5093.859330] LustreError: 159231:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5093.898353] LustreError: 159231:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 5098.981648] LustreError: 159232:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5099.083500] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5099.574710] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5104.480691] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 5113.858564] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5119.314195] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5125.389814] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5125.735898] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5128.822648] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5131.921993] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 5139.774665] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5144.199559] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5153.775682] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 5153.784328] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5153.808080] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5154.796679] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5154.798955] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5154.802887] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5154.820901] Lustre: Skipped 1 previous similar message [ 5159.932607] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5159.941197] Lustre: Skipped 1 previous similar message [ 5160.238259] Lustre: server umount lustre-OST0000 complete [ 5165.028690] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5165.050588] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5165.064070] Lustre: Skipped 1 previous similar message [ 5170.151705] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5170.164978] Lustre: Skipped 1 previous similar message [ 5172.679903] Lustre: server umount lustre-MDT0000 complete [ 5175.273611] LustreError: 159232:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5175.299925] LustreError: 159232:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 5176.540572] LustreError: 162045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790901182 with bad export cookie 2667048009287250189 [ 5176.552334] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5176.561211] LustreError: 162045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5177.089215] Lustre: server umount lustre-MDT0001 complete [ 5188.810570] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [ 5191.612975] Key type lgssc unregistered [ 5191.861701] LNet: 162796:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5191.871939] LNetError: 162796:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5191.885615] LNet: Removed LNI 192.168.201.131@tcp [ 5192.818155] Key type .llcrypt unregistered [ 5192.820051] Key type ._llcrypt unregistered [ 5226.281571] Key type ._llcrypt registered [ 5226.288619] Key type .llcrypt registered [ 5226.537317] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 5227.602861] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5227.782662] alg: No test for adler32 (adler32-zlib) [ 5228.814246] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [ 5229.016636] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [ 5230.744214] Key type lgssc registered [ 5232.064308] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5248.386957] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 20:34:12 (1790901252) [ 5265.979849] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_hostid [ 5278.332055] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 5316.728636] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5316.767910] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5318.103802] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5318.139261] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5318.242494] Lustre: lustre-MDT0000: new disk, initializing [ 5318.340346] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5318.370436] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5323.030300] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 5334.234236] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5334.425359] Lustre: 167090:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5334.529164] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5334.535678] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 5334.543379] Lustre: Skipped 1 previous similar message [ 5334.618987] Lustre: lustre-MDT0001: new disk, initializing [ 5334.698705] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5334.750954] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5334.766759] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5339.061462] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 5349.081437] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5356.117976] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5364.965263] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5365.451840] Lustre: lustre-OST0000: new disk, initializing [ 5365.463115] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5365.471158] Lustre: 169059:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5365.668941] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5371.443615] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5371.456769] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5371.605251] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5373.741935] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 5384.719408] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5397.480436] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5397.483882] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5397.499727] Lustre: Skipped 1 previous similar message [ 5397.500594] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5397.532664] Lustre: Skipped 1 previous similar message [ 5402.607970] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5403.181218] Lustre: server umount lustre-OST0000 complete [ 5411.809991] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5411.829280] LustreError: Skipped 1 previous similar message [ 5411.848594] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5411.880477] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5411.898091] Lustre: Skipped 1 previous similar message [ 5414.876102] Lustre: server umount lustre-MDT0000 complete [ 5418.853838] LustreError: 167085:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790901424 with bad export cookie 15513218015364377696 [ 5418.854613] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5418.870529] LustreError: 167085:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5419.551679] Lustre: server umount lustre-MDT0001 complete [ 5441.420502] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [ 5447.757100] Key type lgssc unregistered [ 5448.159258] LNet: 170986:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5448.178463] LNetError: 170986:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5448.211125] LNet: Removed LNI 192.168.201.131@tcp [ 5449.673362] Key type .llcrypt unregistered [ 5449.679205] Key type ._llcrypt unregistered [ 5483.533333] Key type ._llcrypt registered [ 5483.535963] Key type .llcrypt registered [ 5483.651792] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 5484.916890] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5485.107102] alg: No test for adler32 (adler32-zlib) [ 5486.223934] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [ 5486.712802] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [ 5488.392270] Key type lgssc registered [ 5490.018917] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5499.386799] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5499.405611] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5501.207979] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5507.161714] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 5511.669614] LustreError: 172415:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5511.689516] LustreError: 172415:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 5516.781058] LustreError: 172414:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5517.095944] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5517.492367] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5523.092774] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 5531.684285] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5538.623653] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5547.206419] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5547.622433] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5553.641454] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 5561.327949] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5571.191785] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5578.855434] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5578.880536] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5578.900219] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5578.910865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.921291] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5579.893447] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5579.914428] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5579.935771] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5579.949282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.959021] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5581.002466] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5581.021986] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5581.033701] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5581.038456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.046564] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5581.863331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.777951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.731819] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5583.735864] Lustre: Skipped 2 previous similar messages [ 5583.754373] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5583.764346] Lustre: Skipped 2 previous similar messages [ 5583.776092] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5583.781705] Lustre: Skipped 2 previous similar messages [ 5583.784921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.796337] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5583.802986] Lustre: Skipped 2 previous similar messages [ 5584.760715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.723891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.804543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.676182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.885434] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5588.905405] Lustre: Skipped 4 previous similar messages [ 5588.931901] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5588.967419] Lustre: Skipped 4 previous similar messages [ 5588.984421] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5588.994815] Lustre: Skipped 4 previous similar messages [ 5589.008739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.027143] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5589.041716] Lustre: Skipped 4 previous similar messages [ 5590.392299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.318674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.288474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.293624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.144652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.048497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.863322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.673212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.336347] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5597.339967] Lustre: Skipped 8 previous similar messages [ 5597.351182] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5597.357528] Lustre: Skipped 8 previous similar messages [ 5597.360689] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5597.363578] Lustre: Skipped 8 previous similar messages [ 5597.366750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.376123] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5597.382614] Lustre: Skipped 8 previous similar messages [ 5598.183693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.258165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.052733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.030438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.855273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.785575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.730248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.611711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.491376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.463640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.363888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.263590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.392964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.616628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.871581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.984734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.224034] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5614.235723] Lustre: Skipped 16 previous similar messages [ 5614.270350] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5614.288364] Lustre: Skipped 16 previous similar messages [ 5614.296493] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5614.301917] Lustre: Skipped 16 previous similar messages [ 5614.304557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.319251] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5614.325864] Lustre: Skipped 16 previous similar messages [ 5615.544814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.659396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.812863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.978546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.815184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.747081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.619525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.706933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.577033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.644407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.564278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.577562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.386909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.319716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.290479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.380735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.294022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.144936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.113763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.068790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.092071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.237389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.441182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.506976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.313279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.449809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.741877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.681283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.917583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.837617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.215984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.444301] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5647.447443] Lustre: Skipped 31 previous similar messages [ 5647.475731] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5647.489395] Lustre: Skipped 31 previous similar messages [ 5647.492872] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5647.497766] Lustre: Skipped 31 previous similar messages [ 5647.500860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.519260] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5647.533888] Lustre: Skipped 31 previous similar messages [ 5648.623960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.802036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.949634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.826431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.900295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.151904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.357604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.555123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.475052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.890571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.888642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.289684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.236978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.520143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.646520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.723528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.490125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.097082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.712391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.670513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.736693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.347544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.444970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.920143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.901362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.229294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.553060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.350091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.515156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.814482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.909777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.486494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.459256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.107272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.050916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.963828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.151648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.309847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.895121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.912930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.895313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.334933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.425183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.314493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.242810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.345261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.811787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.923253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.705130] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5712.714793] Lustre: Skipped 48 previous similar messages [ 5712.768033] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5712.784200] Lustre: Skipped 48 previous similar messages [ 5712.800200] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5712.802489] Lustre: Skipped 48 previous similar messages [ 5712.818792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.830362] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5712.848552] Lustre: Skipped 48 previous similar messages [ 5713.979454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.011796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.108030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.262651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.366653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.462868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.569841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.676331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.839943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.681805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.596808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.710390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.981974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.064081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.983381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.155259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.198967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.743752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.219302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.550770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.824457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.271057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.767469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.974155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.492778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.864652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.252458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.435177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.540350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.724533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.819214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.498801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.790258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.526558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.458850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.900170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.311907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.531099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.048446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.129809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.455500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.796785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.108774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.444776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.775419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.115666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.181490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.464249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.822639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.858464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.628398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.634741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.912232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.076911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.046611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.164324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.149472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.174957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.134268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.149711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.196044] LustreError: 172435:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5784.204302] LustreError: 172435:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5784.236733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.376881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.261060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.170633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.216827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.350324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.349071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.381737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.536304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.841559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.861724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.455025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.834278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.106287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.030384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.141690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.382194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.698389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.993740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.542610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.932193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.299511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.090479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.634838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.099757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.629564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.050412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.250255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.490482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.375850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.339030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.290574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.717228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.833538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.923855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.731232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.764163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.672709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.722512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.732440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.813076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.845833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.016492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.922507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.871548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.947083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.891786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.074954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.981549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.038277] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5841.045502] Lustre: Skipped 109 previous similar messages [ 5841.059029] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5841.074128] Lustre: Skipped 109 previous similar messages [ 5841.077927] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5841.081882] Lustre: Skipped 109 previous similar messages [ 5841.085469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.093761] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5841.100976] Lustre: Skipped 109 previous similar messages [ 5842.011591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.047309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.129435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.998853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.918828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.037072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.954932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.904733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.743493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.650444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.531439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.603645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.762579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.728044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.809709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.706685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.894373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.925941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.964053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.911996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.045900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.067689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.151555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.174725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.134421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.183032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.995352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.743439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.773962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.896492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.917379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.922967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.863887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.653746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.644395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.549326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.551833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.480533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.563666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.566297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.679709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.933884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.292935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.381891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.493760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.589192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.764221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.126602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.273752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.500905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.712940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.171642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.384220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.769262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.986567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.232522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.503993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.985914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.907319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.050604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.983317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.837662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.837032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.662648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.618624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.535103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.476745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.507625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.620238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.529448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.558268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.353136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.312085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.231257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.176097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.098581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.945856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.022483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.942454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.864834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.731315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.587493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.605307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.526904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.508421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.365509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.550870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.370359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.455184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.588796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.502768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.549298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.453412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.460254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.422054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.300554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.125285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.083962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.956743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.873931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.768410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.696652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.648494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.684744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.636059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.666448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.390560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.363850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.207228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.215961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.231877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.155342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.132270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.406564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.798018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.982703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.918337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.903682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.929811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.020197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.884629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.697933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.558261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.507618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.485510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.423474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.210291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.173896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.116451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.145402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.164241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.992594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.003641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.738137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.718782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.678112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.645779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.618821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.605500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.384905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.266115] LustreError: 172435:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5982.282065] LustreError: 172435:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5982.299837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.239521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.319843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.501489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.346007] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.355815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.346923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.392480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.435119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.465331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.455715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.467817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.504749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.346658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.247657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.174454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.175948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.069027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.125069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.002464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.996903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.962985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.993874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.003566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.988727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.073629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.839529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.684066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.833351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.612315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.594411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.600497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.710459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.813513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.798062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.863062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.948391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.908772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.110156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.216252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.157816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.208551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.125721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.053195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.989412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.020113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.764185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.689978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.624722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.746224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.623050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.702247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.685361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.668415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.420691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.326495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.179948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.142545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.102317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.047142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.938262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.917564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.932913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.934659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.786836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.739664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.494987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.555507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.464472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.307365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.144606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.161186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.063968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.066205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.079557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.345801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.508677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.695148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.855277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.378882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.343716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.498734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.751919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.802961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.901956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.124115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.310878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.314737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.307650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.122743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.086758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.909332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.722367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.457921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.399495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.312368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.532517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.499833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.691389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.828785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.675970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.908779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.887307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.061874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.968511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.944707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.960390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.934961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.966335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.031911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.838511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.971148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.012191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.887711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.772836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.013949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.173064] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6098.178763] Lustre: Skipped 256 previous similar messages [ 6098.201438] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6098.228344] Lustre: Skipped 256 previous similar messages [ 6098.248308] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6098.254898] Lustre: Skipped 256 previous similar messages [ 6098.258944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.267463] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6098.282407] Lustre: Skipped 256 previous similar messages [ 6099.213740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.215233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.286526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.098722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.356818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.363150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.435307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.933326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.077955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.163597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.347216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.616169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.881874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.112960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.175683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.120278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.301073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.281084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.388249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.347287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.563055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.692541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.598125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.934920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.924757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.914455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.061089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.067136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.169581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.155497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.403786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.608640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.660314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.426183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.813884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.282562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.094900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.001094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.982423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.893082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.942744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.880536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.764079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.807544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.719903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.843443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.690962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.679247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.550220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.458465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.282155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.237503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.000204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.846280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.650868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.396238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.224938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.108941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.851708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.638855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.360114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.113510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.004035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.883638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.971676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.083134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.137391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.102718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.107814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.948118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.819101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.597584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.531081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.236477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.016912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.848680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.804219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.703506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.485928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.292695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.153065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.173669] LustreError: 172435:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6179.186190] LustreError: 172435:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6179.200291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.120370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.061440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.094827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.051490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.861742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.856400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.805692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.793693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.887101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.834780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.852836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.817138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.909915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.773646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.714645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.482064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.565350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.297401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.023979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.748135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.661675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.432919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.259463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.073410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.000753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.905856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.001687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.017636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.828759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.598150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.733484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.540805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.522571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.399940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.250124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.309490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.418733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.573040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.650596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.605762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.511464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.433976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.490632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.589648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.586313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.674567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.746933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.711125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.614636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.612865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.973994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.150117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.171982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.503570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.924632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.403896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.991701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.475928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.579510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.750220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.394788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.485440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.816299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.026450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.229737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.683759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.839951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.117314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.605198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.121159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.062758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.391592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.736853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.999381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.715184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.726058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.733198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.858217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.882586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.877605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.977384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.066203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.958425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.711524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.605493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.424216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.478060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.289889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.090749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.116848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.016851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.872039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.680031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.672761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.502996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.304241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.382565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.289076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.282813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.306633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.364310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.150788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.128908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.064720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.035760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.899384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.879721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.983726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.122227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.079889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.213676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.288285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.363947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.483994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.456438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.411967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.311614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.406813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.305324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.380957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.333954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.283871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.192126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.222983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.044586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.866705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.825965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.860421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.941962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.034200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.041840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.107475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.311558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.464630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.569967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.739980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.852676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.010351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.825919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.631628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.514182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.362774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.280855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.423622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.569392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.438693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.273344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.208334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.096350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.308723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.254919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.213083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.990795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.005180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.849852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.744322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.765784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.677953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.716581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.616940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.661218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.598994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.688978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.656669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.802666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.693164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.764211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.747477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.821644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.729152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.796344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.708881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.740928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.649592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.614793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.462854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.432169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.650803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.709757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.754917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.699837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.539682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.600360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.545371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.793895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.568224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.418608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.234815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.407815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.347859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.449430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.770968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.990168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.130990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.340224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.388805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.349749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.297375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.136977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.151678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.105658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.902687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.892218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.960533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.015207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.959613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.864572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.929565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.801966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.791285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.666499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.621045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.814103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.728133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.806816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.701702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.686258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.547512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.529222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.446288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.329983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.077223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.903812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.663819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.471709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.498289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.720734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.812589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.726393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.779598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.497177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.236714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.185439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.217556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.186582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.436125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.562993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.483562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.523966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.413561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.413526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.206412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.245317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.169794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.377110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.558880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.779375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.809776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.016094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.933399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.869084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.875421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.045825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.860666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.815111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.609298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.815861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.963874] LustreError: 172435:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6438.973442] LustreError: 172435:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6438.982970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.125866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.397213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.711231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.665651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.741936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.871911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.753715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.897889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.988837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.975905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.952938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.106839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.193648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.250811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.225249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.245329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.410922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.369570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.601503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.805311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.939162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.129243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.667721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.563597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.470314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.331787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.370427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.368596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.395282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.589647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.593406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.721306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.726245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.803987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.760235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.845139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.068922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.359795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.547101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.605522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.603996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.714562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.624672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6486.726853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.545831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.564701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.769574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.133040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.270342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.354282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.525864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.675879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.764885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.770580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.947443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.132654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.265838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.912194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.869695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.848171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.976078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.750395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.745357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.597126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.548231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.518735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.312357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.287282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.339490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.235682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.238329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.067591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.107355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.056260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.170862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.280563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6520.234891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.326249] LustreError: 172435:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6521.335882] LustreError: 172435:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6521.345805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.370621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.440271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.576879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.684902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6526.733592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.771655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.714768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6529.819768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.006813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.955155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.938169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.849887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6534.670666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.555761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.243185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.221411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6538.128608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.070738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.013634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.871920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.821958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.885714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.909368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.835567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6545.881343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.998084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.113858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.158183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.112503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.247045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6552.228412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.049939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.905387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.731390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.547165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.561507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.484439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.573901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.568813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.421624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.293568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.190190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.058359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.039023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.895403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.656282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.513109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.405712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.331606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.032899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.891600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.539046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.331598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.975404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.815086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.604041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.537728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.355286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.376202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.178793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.302467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.289228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.182575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.137710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.192266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.152635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.145581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.998262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.876674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6586.877738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.852082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.895586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.830067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.776687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.710392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6592.672993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6593.489539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.207668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.151312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.165579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.212275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.040511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.029932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.923752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.925580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.900362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.935616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.976364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.941608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6606.070103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6607.013953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.103989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.033906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.083341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.196603] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6611.202504] Lustre: Skipped 512 previous similar messages [ 6611.220750] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6611.234630] Lustre: Skipped 512 previous similar messages [ 6611.241445] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6611.244555] Lustre: Skipped 512 previous similar messages [ 6611.248560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.258320] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6611.270837] Lustre: Skipped 512 previous similar messages [ 6612.110639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.333192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7041.662961] LustreError: 174910:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7041.669435] LustreError: 172435:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7719.911086] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7719.936316] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7719.953533] Lustre: Skipped 3 previous similar messages [ 7725.035308] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7725.051549] Lustre: Skipped 2 previous similar messages [ 7726.117871] Lustre: server umount lustre-OST0000 complete [ 7734.242843] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7734.263729] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7737.430352] Lustre: server umount lustre-MDT0000 complete [ 7740.390311] LustreError: 173154:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7740.402993] LustreError: 173154:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 7740.626404] LustreError: 172398:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790903746 with bad export cookie 4920506479634295275 [ 7740.630614] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7740.637532] LustreError: 172398:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7741.221419] Lustre: server umount lustre-MDT0001 complete [ 7753.693169] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [ 7756.620952] Key type lgssc unregistered [ 7757.050174] LNet: 227723:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7757.060198] LNetError: 227723:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7758.125739] LNet: Removed LNI 192.168.201.131@tcp [ 7759.169556] Key type .llcrypt unregistered [ 7759.171321] Key type ._llcrypt unregistered [ 7789.379600] Key type ._llcrypt registered [ 7789.382392] Key type .llcrypt registered [ 7789.592576] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 7790.590316] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7790.697625] alg: No test for adler32 (adler32-zlib) [ 7791.670757] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [ 7791.842473] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [ 7793.480211] Key type lgssc registered [ 7794.951957] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7805.444884] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7805.491044] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7807.294255] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7812.867137] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 7817.700551] LustreError: 229155:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7817.728744] LustreError: 229155:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 7822.821842] LustreError: 229156:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7824.526869] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7825.010145] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7831.371981] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 7840.597943] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7847.721560] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7855.987256] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7856.419508] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7864.782529] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 7876.159467] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7876.960286] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7881.844482] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7884.316681] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7884.333124] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7884.343927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7884.365059] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7884.365622] LustreError: 229175:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7887.332588] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7887.363229] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7888.775538] Lustre: lustre-OST0000: Not available for connect from 192.168.201.31@tcp (stopping) [ 7888.785235] Lustre: Skipped 1 previous similar message [ 7892.453781] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7892.457029] Lustre: Skipped 1 previous similar message [ 7892.885404] Lustre: server umount lustre-OST0000 complete [ 7904.432172] LustreError: 229175:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7928.323654] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7928.666680] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7935.130754] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 7941.991578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7942.006603] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7942.011831] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7942.030552] Lustre: Skipped 1 previous similar message [ 7944.437826] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7946.912705] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7954.103189] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9fae474b4800.ost_server_uuid 50 [ 7955.734331] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9fae474b4800.ost_server_uuid in FULL state after 0 sec [ 7961.060789] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7961.073420] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7961.082674] Lustre: Skipped 1 previous similar message [ 7961.091285] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7962.605126] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7967.380615] Lustre: server umount lustre-OST0000 complete [ 7972.833252] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7972.848854] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7972.849922] Lustre: Skipped 2 previous similar messages [ 7972.867021] Lustre: Skipped 2 previous similar messages [ 7982.050427] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7982.065691] Lustre: Skipped 4 previous similar messages [ 7986.147489] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7986.382915] Lustre: server umount lustre-MDT0000 complete [ 7987.172181] LustreError: 229150:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7987.201191] LustreError: 229150:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 7991.813686] LustreError: 229138:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790903997 with bad export cookie 3865639803754040465 [ 7991.825600] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7992.603852] Lustre: server umount lustre-MDT0001 complete [ 8011.259677] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [ 8014.231806] Key type lgssc unregistered [ 8014.468918] LNet: 234364:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8014.477892] LNetError: 234364:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8014.504947] LNet: Removed LNI 192.168.201.131@tcp [ 8015.463320] Key type .llcrypt unregistered [ 8015.467262] Key type ._llcrypt unregistered [ 8053.361963] Key type ._llcrypt registered [ 8053.364616] Key type .llcrypt registered [ 8053.454178] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 8055.123050] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8055.161775] alg: No test for adler32 (adler32-zlib) [ 8056.296359] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [ 8056.542913] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [ 8058.272217] Key type lgssc registered [ 8059.502987] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8070.633936] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8070.665852] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8072.280880] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8078.545890] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 8082.416393] LustreError: 236115:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8082.447819] LustreError: 236115:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 8087.524669] LustreError: 236114:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8089.221557] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8089.647817] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8096.175902] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 8100.488331] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8110.157764] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8110.613690] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8117.815765] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 8118.780738] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 8128.856100] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8129.015204] Lustre: lustre-OST0001: new disk, initializing [ 8129.019167] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8129.024082] Lustre: 238426:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8129.119876] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8137.194397] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 8138.846542] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8138.855672] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8139.015685] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8148.875687] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8157.246871] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8175.102558] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8175.111681] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8175.133712] Lustre: Skipped 1 previous similar message [ 8175.156031] Lustre: Skipped 3 previous similar messages [ 8179.868562] Lustre: server umount lustre-MDT0000 complete [ 8180.205600] LustreError: 236843:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8180.222267] LustreError: 236843:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 8184.968988] LustreError: 236096:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790904191 with bad export cookie 5132788173702844550 [ 8184.984613] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8184.988354] LustreError: 236096:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8185.313768] LustreError: 237219:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8185.322320] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8185.335280] LustreError: 237219:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 8185.383212] Lustre: Skipped 3 previous similar messages [ 8185.539586] Lustre: server umount lustre-MDT0001 complete [ 8191.715440] Lustre: server umount lustre-OST0000 complete [ 8196.254859] Lustre: server umount lustre-OST0001 complete [ 8221.778195] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 8232.994912] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8233.389505] LustreError: 241534:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8233.479345] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8238.206397] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 8243.685309] LustreError: 241535:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8243.709250] LustreError: 241535:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 8247.686397] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8248.063876] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8253.467831] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 8256.912229] Lustre: 242676:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8265.114193] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8265.595348] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8272.634991] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 8273.808064] LustreError: 243031:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8273.826097] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 8273.835104] LustreError: 243031:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 8278.990372] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 8283.360598] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8290.658370] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 8299.653682] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8304.323339] Lustre: 244549:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8319.461601] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8319.469980] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8319.485162] Lustre: Skipped 3 previous similar messages [ 8319.503993] Lustre: Skipped 3 previous similar messages [ 8322.560859] Lustre: server umount mylustre-MDT0000 complete [ 8324.591192] LustreError: 241535:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8324.623679] LustreError: 241535:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 8327.253061] LustreError: 241511:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790904333 with bad export cookie 5132788173702847413 [ 8327.264543] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8327.270157] LustreError: 241511:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8327.686555] Lustre: server umount mylustre-MDT0001 complete [ 8343.340633] Lustre: server umount mylustre-OST0000 complete [ 8344.928128] Lustre: 235367:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790904335/real 1790904335] req@ffff8ab6697a8a80 x1877899017883392/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790904351 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8344.952899] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8347.924957] Lustre: server umount mylustre-OST0001 complete [ 8374.315451] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 8386.892224] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8387.866610] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8387.871133] Lustre: Skipped 1 previous similar message [ 8392.773861] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 8393.190938] LustreError: 246860:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8393.218552] LustreError: 246860:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 8402.860902] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8408.909526] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 8411.917943] Lustre: 248002:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8419.624587] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8427.930216] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 8428.283221] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:161) [ 8432.405294] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 8438.530119] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8443.913071] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 8447.000707] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 8456.274531] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8460.551409] Lustre: 249872:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8469.474677] Lustre: tfs-MDT0000-lwp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8469.486916] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8469.490436] Lustre: Skipped 3 previous similar messages [ 8469.500066] Lustre: Skipped 3 previous similar messages [ 8474.611614] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8474.622814] Lustre: Skipped 3 previous similar messages [ 8475.147311] Lustre: server umount tfs-MDT0000 complete [ 8479.930362] LustreError: 246834:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790904486 with bad export cookie 5132788173702849366 [ 8479.945054] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8479.947958] LustreError: 246834:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8480.464842] Lustre: server umount tfs-MDT0001 complete [ 8495.549282] Lustre: server umount tfs-OST0000 complete [ 8500.194658] Lustre: 235369:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790904490/real 1790904490] req@ffff8ab664377100 x1877899017939712/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790904506 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8500.229702] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8500.931710] Lustre: server umount tfs-OST0001 complete [ 8529.654637] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 8543.237424] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8543.808028] LustreError: 252125:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8543.822779] LustreError: 252125:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 11 previous similar messages [ 8543.923581] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8543.934742] Lustre: Skipped 3 previous similar messages [ 8549.293301] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 8559.512362] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8565.587621] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 8569.528556] Lustre: 253267:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8579.327821] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8584.079734] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:193) [ 8588.166782] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 8588.263214] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 8598.999538] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8604.693159] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 8608.198345] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 8618.314375] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8622.815466] Lustre: 255140:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8632.410883] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 21:30:36 (1790904636) [ 8650.726581] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8650.738198] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8650.745584] Lustre: Skipped 3 previous similar messages [ 8650.751983] Lustre: Skipped 3 previous similar messages [ 8656.790355] Lustre: server umount lustre-MDT0000 complete [ 8662.058972] LustreError: 253269:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790904668 with bad export cookie 5132788173702851284 [ 8662.083909] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8662.103505] LustreError: 253269:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8666.084333] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8666.106610] Lustre: Skipped 5 previous similar messages [ 8669.111523] Lustre: server umount lustre-MDT0001 complete [ 8680.148975] Lustre: server umount lustre-OST0000 complete [ 8699.360170] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8699.766166] Lustre: server umount lustre-OST0001 complete [ 8709.318759] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_hostid [ 8722.396450] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 8764.615718] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8765.097665] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8765.167360] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8765.284516] Lustre: lustre-MDT0000: new disk, initializing [ 8765.378477] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8765.389625] Lustre: Skipped 3 previous similar messages [ 8765.411113] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8771.244150] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 8785.320396] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8785.669434] Lustre: 258329:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8785.793039] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8785.810227] Lustre: Skipped 1 previous similar message [ 8786.026821] Lustre: lustre-MDT0001: new disk, initializing [ 8786.280067] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8786.308334] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8793.331809] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 8807.269737] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8807.743633] Lustre: lustre-OST0000: new disk, initializing [ 8807.762757] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8807.777458] Lustre: 259914:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8809.747645] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8809.772476] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8809.977577] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8816.145558] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 8829.079328] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8829.213086] Lustre: lustre-OST0001: new disk, initializing [ 8829.215947] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8829.220516] Lustre: 260785:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8831.121645] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8831.131457] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8831.218886] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8836.636188] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 8851.885520] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 21:34:15 (1790904855) [ 8867.534468] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 21:34:31 (1790904871) [ 8886.753872] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8886.760878] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8886.770427] LustreError: Skipped 1 previous similar message [ 8886.772618] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8886.798750] Lustre: Skipped 3 previous similar messages [ 8886.845514] Lustre: Skipped 3 previous similar messages [ 8890.200307] Lustre: server umount lustre-OST0000 complete [ 8892.933898] LustreError: 259899:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8892.959869] LustreError: 259899:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 19 previous similar messages [ 8901.548669] Lustre: server umount lustre-MDT0000 complete [ 8906.182809] LustreError: 258323:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790904912 with bad export cookie 5132788173702853055 [ 8906.183695] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8906.193680] LustreError: 258323:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8912.947419] Lustre: server umount lustre-MDT0001 complete [ 8925.804263] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [ 9003.089875] Lustre: server umount lustre-OST0001 complete [ 9011.543404] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_hostid [ 9029.617041] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 9078.100455] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9078.464367] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9078.500063] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9078.651202] Lustre: lustre-MDT0000: new disk, initializing [ 9078.797085] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9078.800527] Lustre: Skipped 3 previous similar messages [ 9078.843232] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9086.675426] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 9100.860886] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9100.991923] Lustre: 265643:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9101.061749] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9101.063904] Lustre: Skipped 1 previous similar message [ 9101.144565] Lustre: lustre-MDT0001: new disk, initializing [ 9101.341877] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9101.356354] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9106.663238] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 9119.905933] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9128.559985] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9137.741744] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9138.105751] Lustre: lustre-OST0000: new disk, initializing [ 9138.113847] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9138.125210] Lustre: 267614:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9139.567116] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9139.585404] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9139.817856] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9147.094236] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 9160.503382] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9170.401347] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9170.412931] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9170.416715] LustreError: Skipped 1 previous similar message [ 9170.425967] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9170.447489] Lustre: Skipped 5 previous similar messages [ 9170.486143] Lustre: Skipped 8 previous similar messages [ 9175.243410] Lustre: server umount lustre-OST0000 complete [ 9186.940590] Lustre: server umount lustre-MDT0000 complete [ 9191.233409] LustreError: 265635:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790905197 with bad export cookie 5132788173702854532 [ 9191.253681] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9191.839660] Lustre: server umount lustre-MDT0001 complete [ 9212.001216] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [ 9216.368167] Key type lgssc unregistered [ 9216.687646] LNet: 269396:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9216.695433] LNetError: 269396:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9216.719420] LNet: Removed LNI 192.168.201.131@tcp [ 9217.850183] Key type .llcrypt unregistered [ 9217.853093] Key type ._llcrypt unregistered [ 9252.827456] Key type ._llcrypt registered [ 9252.829844] Key type .llcrypt registered [ 9252.976771] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 9254.814602] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9255.072814] alg: No test for adler32 (adler32-zlib) [ 9256.235765] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [ 9256.693823] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [ 9258.426866] Key type lgssc registered [ 9260.860633] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9272.555480] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9272.579737] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9274.623573] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9279.727117] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 9284.083331] LustreError: 270876:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9284.111670] LustreError: 270876:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 9289.188840] LustreError: 270877:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9291.093942] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9291.815060] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9299.623962] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 9311.455962] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9319.838627] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9328.653496] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9329.001162] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9336.652818] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 9344.387773] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 9346.186502] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9347.500428] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9351.650595] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9351.660937] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9351.687088] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9354.725530] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9354.733098] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9354.761371] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9356.410723] Lustre: server umount lustre-OST0000 complete [ 9363.435090] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9363.457238] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9363.488633] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9363.491660] Lustre: Skipped 1 previous similar message [ 9370.082126] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9370.101513] Lustre: Skipped 3 previous similar messages [ 9375.200307] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9375.208548] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9375.231384] Lustre: Skipped 1 previous similar message [ 9375.578136] Lustre: server umount lustre-MDT0000 complete [ 9379.314525] LustreError: 270876:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9379.350337] LustreError: 270876:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9379.978569] LustreError: 270858:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790905386 with bad export cookie 17411860286318643989 [ 9380.000865] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9380.465373] Lustre: server umount lustre-MDT0001 complete [ 9394.021826] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [ 9397.032989] Key type lgssc unregistered [ 9397.306914] LNet: 274465:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9397.318948] LNetError: 274465:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9397.334482] LNet: Removed LNI 192.168.201.131@tcp [ 9398.519249] Key type .llcrypt unregistered [ 9398.521701] Key type ._llcrypt unregistered [ 9419.218559] Key type ._llcrypt registered [ 9419.221286] Key type .llcrypt registered [ 9419.418987] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 9421.407367] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9421.781249] alg: No test for adler32 (adler32-zlib) [ 9422.880497] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [ 9423.359556] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [ 9425.043738] Key type lgssc registered [ 9427.202338] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9441.959365] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9441.986178] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9457.632687] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9462.752795] LustreError: 275675:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.131@tcp: failed processing log, type 4: rc = -110 [ 9489.504287] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9496.735911] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 9505.856842] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9508.599612] Lustre: server umount lustre-OST0000 complete [ 9529.574538] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [ 9532.931650] Key type lgssc unregistered [ 9533.284840] LNet: 277308:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9533.293294] LNetError: 277308:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9533.312991] LNet: Removed LNI 192.168.201.131@tcp [ 9534.088207] Key type .llcrypt unregistered [ 9534.091576] Key type ._llcrypt unregistered [ 9578.389176] Key type ._llcrypt registered [ 9578.396685] Key type .llcrypt registered [ 9578.623876] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 9580.353163] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9580.461069] alg: No test for adler32 (adler32-zlib) [ 9581.554283] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [ 9581.792641] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [ 9583.459260] Key type lgssc registered [ 9585.304953] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9602.175289] loop0: detected capacity change from 0 to 204800000 [ 9602.440580] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9602.455331] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9602.940365] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9604.398562] loop0: detected capacity change from 0 to 204800000 [ 9604.701525] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9604.720455] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9605.403717] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9606.675316] loop0: detected capacity change from 0 to 204800000 [ 9607.031497] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9607.055759] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9607.275357] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9608.758745] loop0: detected capacity change from 0 to 204800000 [ 9608.960316] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9608.986935] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9609.156161] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9610.557986] loop0: detected capacity change from 0 to 204800000 [ 9610.624233] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9617.027866] loop0: detected capacity change from 0 to 204800000 [ 9617.098676] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9622.259936] loop0: detected capacity change from 0 to 204800000 [ 9622.303787] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9627.423413] loop0: detected capacity change from 0 to 204800000 [ 9627.481965] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9632.586912] loop0: detected capacity change from 0 to 204800000 [ 9632.721044] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9632.810271] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9632.927192] Lustre: 280739:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9632.941050] Lustre: 280739:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9632.953488] Lustre: 280739:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9632.966120] Lustre: 280739:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9633.014100] Lustre: 280739:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9633.039507] Lustre: 280739:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9633.053186] Lustre: 280739:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9633.076737] Lustre: 280739:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9633.088109] Lustre: 280739:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9633.100738] Lustre: 280739:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9633.118197] Lustre: 280739:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9633.131771] Lustre: 280739:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9633.151812] Lustre: 280739:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9633.173967] Lustre: 280739:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9640.499757] Lustre: server umount MGS complete [ 9641.646603] loop0: detected capacity change from 0 to 204800000 [ 9641.784777] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9642.474877] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9642.599250] Lustre: lustre-MDT0000: Aborting client recovery [ 9643.895447] loop1: detected capacity change from 0 to 204800000 [ 9644.208602] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9644.374182] Lustre: 281171:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9644.398267] Lustre: 281171:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9644.425889] Lustre: 281171:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9644.441584] Lustre: 281171:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9644.455576] Lustre: 281171:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9644.467603] Lustre: 281171:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9644.480746] Lustre: 281171:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9644.499594] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9644.739895] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9644.770483] Lustre: lustre-MDT0001: Aborting client recovery [ 9645.800412] loop2: detected capacity change from 0 to 204800000 [ 9645.981656] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9646.050092] Lustre: 281253:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9646.059565] Lustre: 281253:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9646.070508] Lustre: 281253:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9646.079867] Lustre: 281253:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9646.099910] Lustre: 281253:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9646.128440] Lustre: 281253:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9646.164757] Lustre: 281253:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9646.295830] Lustre: Found index 0 for lustre-OST0000, updating log [ 9646.328442] Lustre: 281272:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9646.385514] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9646.398101] Lustre: lustre-OST0000: Aborting recovery [ 9647.443498] loop3: detected capacity change from 0 to 204800000 [ 9647.552722] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9647.678821] Lustre: 281338:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9647.697649] Lustre: 281338:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9647.727695] Lustre: 281338:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9647.741483] Lustre: 281338:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9647.749371] Lustre: 281338:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9647.760945] Lustre: 281338:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9647.773987] Lustre: 281338:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9647.819799] Lustre: Found index 1 for lustre-OST0001, updating log [ 9647.893832] Lustre: 281345:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9648.012980] Lustre: lustre-OST0001: Aborting recovery [ 9652.069301] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9652.073188] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9652.096617] LustreError: 281115:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 9652.149859] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9652.159952] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9654.122603] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9654.123401] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9656.261224] LustreError: 281082:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 3256us: rc = -2 [ 9656.776620] LustreError: 281083:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0001: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0001 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0001/identity_upcall, time 2216us: rc = -2 [ 9656.800221] LustreError: 281083:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 14 previous similar messages [ 9670.626070] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9670.632857] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9670.662691] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9674.721951] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9674.728465] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9674.753494] Lustre: Skipped 1 previous similar message [ 9676.876893] Lustre: server umount lustre-MDT0000 complete [ 9677.834704] LustreError: 281064:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790905683 with bad export cookie 4379212915721638149 [ 9677.851154] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9677.851906] LustreError: 281064:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9679.843199] LustreError: 281655:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9679.843819] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9679.849774] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9679.849785] Lustre: Skipped 3 previous similar messages [ 9679.861092] LustreError: 281655:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 9679.874325] Lustre: Skipped 2 previous similar messages [ 9682.859521] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9682.875417] LustreError: 281080:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9682.879316] Lustre: Skipped 2 previous similar messages [ 9682.915243] LustreError: 281080:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9684.723809] Lustre: server umount lustre-MDT0001 complete [ 9700.320122] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9700.546661] Lustre: server umount lustre-OST0000 complete [ 9716.192832] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9716.437107] Lustre: server umount lustre-OST0001 complete [ 9726.069484] Lustre: DEBUG MARKER: == conf-sanity test 108c: mkfs.lustre --mmp option enables MMP feature ========================================================== 21:48:50 (1790905730) [ 9754.024517] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 9798.293247] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9798.648924] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9798.665432] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9798.747594] Lustre: lustre-MDT0000: new disk, initializing [ 9798.875679] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9798.881858] Lustre: Skipped 1 previous similar message [ 9798.909908] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9802.989206] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 9811.419494] Lustre: server umount lustre-MDT0000 complete [ 9832.557644] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 21:50:36 (1790905836) [ 9863.861635] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_hostid [ 9874.313470] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [ 9915.162656] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9915.470159] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9915.532157] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9915.545463] Lustre: Skipped 1 previous similar message [ 9915.679302] Lustre: lustre-MDT0000: new disk, initializing [ 9915.868721] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9915.905380] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9921.901920] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 9934.298496] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9934.507476] Lustre: 288453:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9934.619499] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9934.630551] Lustre: Skipped 1 previous similar message [ 9934.771694] Lustre: lustre-MDT0001: new disk, initializing [ 9934.908810] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9934.970676] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9934.999874] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9940.196602] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 9951.137372] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9958.488346] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9966.441391] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9966.941957] Lustre: lustre-OST0000: new disk, initializing [ 9966.946192] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9966.952808] Lustre: 290424:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9967.082265] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9974.314906] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [ 9975.492224] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9975.504487] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9975.630629] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9985.870807] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9993.046869] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10007.231696] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10013.511580] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10067.941825] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10067.947505] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10067.963053] LustreError: Skipped 1 previous similar message [10067.986865] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10068.009928] Lustre: Skipped 1 previous similar message [10068.042770] Lustre: Skipped 1 previous similar message [10071.512381] Lustre: server umount lustre-OST0000 complete [10078.180577] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10078.185290] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10078.215773] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10078.230473] Lustre: Skipped 1 previous similar message [10083.085892] Lustre: server umount lustre-MDT0000 complete [10083.302554] LustreError: 288463:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10083.335151] LustreError: 288463:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10088.289673] LustreError: 291676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790906094 with bad export cookie 4379212915721641117 [10088.297190] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10088.300789] LustreError: 291676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10088.825403] Lustre: server umount lustre-MDT0001 complete [10099.187203] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10105.230435] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [10111.738685] Lustre: server umount MGS complete [10124.852716] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10125.385063] LustreError: 294297:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10125.413415] LustreError: 294310:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10125.491040] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10130.878314] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [10130.915769] LustreError: 294298:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10140.135716] LustreError: 294297:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10140.161851] LustreError: 294297:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10140.225625] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10145.809954] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [10154.307791] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10160.634871] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10168.297532] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10175.854079] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [10184.720649] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10215.907736] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10215.916202] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10215.920436] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10215.927740] Lustre: Skipped 2 previous similar messages [10220.032096] Lustre: server umount lustre-OST0000 complete [10231.279054] LustreError: 296139:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10231.309796] LustreError: 296139:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10231.402687] Lustre: server umount lustre-MDT0000 complete [10237.310461] LustreError: 294277:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790906243 with bad export cookie 4379212915721643070 [10237.313517] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10237.323228] LustreError: 294277:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10237.751487] Lustre: server umount lustre-MDT0001 complete [10251.588549] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [10254.315299] Key type lgssc unregistered [10254.679219] LNet: 298242:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10254.694861] LNetError: 298242:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10254.726288] LNet: Removed LNI 192.168.201.131@tcp [10255.784900] Key type .llcrypt unregistered [10255.787274] Key type ._llcrypt unregistered [10297.198799] Key type ._llcrypt registered [10297.201989] Key type .llcrypt registered [10297.367555] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_hostid [10315.555391] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [10317.352584] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10317.426423] alg: No test for adler32 (adler32-zlib) [10318.550345] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [10318.747273] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [10320.440212] Key type lgssc registered [10322.043963] Lustre: Echo OBD driver; http://www.lustre.org/ [10362.614195] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10362.645767] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10363.999701] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10364.049744] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10364.194500] Lustre: lustre-MDT0000: new disk, initializing [10364.320437] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10364.356571] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10370.286082] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [10382.431948] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10382.565596] Lustre: 302221:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10382.652243] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10382.658483] Lustre: Skipped 1 previous similar message [10382.783772] Lustre: lustre-MDT0001: new disk, initializing [10382.927739] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10382.985737] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10383.014162] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10388.585122] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [10401.357129] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10409.170948] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10418.120872] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10418.514834] Lustre: lustre-OST0000: new disk, initializing [10418.524536] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10418.542287] Lustre: 304192:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10418.650217] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10425.585568] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [10427.984287] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10427.994682] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10428.168382] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10435.501513] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10442.901790] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10455.562678] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10457.617652] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10469.655236] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10515.425859] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10515.436274] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10515.438545] LustreError: Skipped 1 previous similar message [10515.441165] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10515.471433] Lustre: Skipped 1 previous similar message [10517.714968] Lustre: server umount lustre-OST0000 complete [10525.673686] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10525.677711] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10525.694518] Lustre: Skipped 1 previous similar message [10525.713491] Lustre: Skipped 2 previous similar messages [10529.490675] Lustre: server umount lustre-MDT0000 complete [10533.946616] LustreError: 302216:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790906540 with bad export cookie 6826253464765975896 [10533.950554] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10533.956646] LustreError: 302216:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10534.407827] Lustre: server umount lustre-MDT0001 complete [10545.242220] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10550.302303] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [10555.261062] Lustre: server umount MGS complete [10567.017907] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10567.351438] LustreError: 307996:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10567.366992] LustreError: 307996:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [10567.382223] LustreError: 308009:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10567.451539] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10572.651689] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [10572.778734] LustreError: 307997:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10576.871208] LustreError: 307996:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10581.607470] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10581.912112] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10586.817917] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [10594.605859] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10600.859419] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10608.910594] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10609.248447] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10615.773590] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [10622.841871] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10662.886871] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10662.922967] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10662.930620] Lustre: Skipped 1 previous similar message [10665.708567] Lustre: server umount lustre-OST0000 complete [10673.121783] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10673.128120] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10673.132727] Lustre: Skipped 2 previous similar messages [10673.144222] Lustre: Skipped 1 previous similar message [10676.014974] Lustre: server umount lustre-MDT0000 complete [10678.242365] LustreError: 307992:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10678.272834] LustreError: 307992:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10681.095041] LustreError: 307977:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790906687 with bad export cookie 6826253464765977842 [10681.100404] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10681.108545] LustreError: 307977:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10681.665676] Lustre: server umount lustre-MDT0001 complete [10695.570352] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [10699.644860] Key type lgssc unregistered [10700.051178] LNet: 311944:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10700.077541] LNetError: 311944:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10701.093893] LNet: Removed LNI 192.168.201.131@tcp [10702.109185] Key type .llcrypt unregistered [10702.111342] Key type ._llcrypt unregistered [10728.036773] Key type ._llcrypt registered [10728.039254] Key type .llcrypt registered [10728.189626] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_hostid [10743.284524] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [10744.959094] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10745.016842] alg: No test for adler32 (adler32-zlib) [10745.990609] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [10746.169338] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [10747.808969] Key type lgssc registered [10748.789917] Lustre: Echo OBD driver; http://www.lustre.org/ [10774.587230] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [10789.192778] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10789.230854] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10790.461596] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10790.482411] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10790.543195] Lustre: lustre-MDT0000: new disk, initializing [10790.610487] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10790.626225] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10794.767315] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [10805.362407] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10805.624681] Lustre: lustre-OST0000: new disk, initializing [10805.627956] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10805.634363] Lustre: Skipped 1 previous similar message [10805.641713] Lustre: 316009:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10805.699220] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10812.643495] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [10815.105364] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10815.118767] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10815.147207] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [11695.587357] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11695.609133] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11700.677965] Lustre: server umount lustre-MDT0000 complete [11704.968616] LustreError: 315195:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790907711 with bad export cookie 4558099093883268296 [11704.985809] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11705.179261] Lustre: server umount lustre-OST0000 complete [11732.948244] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_hostid [11744.803962] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [11784.046383] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11784.407081] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11784.451974] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11784.542654] Lustre: lustre-MDT0000: new disk, initializing [11784.607190] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11784.630854] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11788.348933] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [11798.714546] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11798.838876] Lustre: 320350:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11798.913467] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11798.920470] Lustre: Skipped 1 previous similar message [11799.014781] Lustre: lustre-MDT0001: new disk, initializing [11799.132993] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11799.167723] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11799.181849] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11803.764120] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [11812.632387] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11819.114712] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11826.473564] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11826.723312] Lustre: lustre-OST0000: new disk, initializing [11826.726896] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11826.740354] Lustre: 322317:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11826.830550] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11833.264196] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [11833.963505] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11833.983960] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11834.048971] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11843.240892] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11848.485677] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11848.710751] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11853.971706] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11854.197566] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11859.940522] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11859.948542] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11859.956715] Lustre: Skipped 1 previous similar message [11859.957507] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11859.981274] LustreError: Skipped 1 previous similar message [11862.431313] Lustre: server umount lustre-OST0000 complete [11870.180200] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11870.181822] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11870.194914] Lustre: Skipped 1 previous similar message [11870.209222] Lustre: Skipped 2 previous similar messages [11873.500840] Lustre: server umount lustre-MDT0000 complete [11878.501830] LustreError: 320343:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790907884 with bad export cookie 4558099093886209115 [11878.524160] LustreError: 320343:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11878.525693] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11879.290226] Lustre: server umount lustre-MDT0001 complete [11883.953916] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11885.977679] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 22:24:49 (1790907889) [11893.362886] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11893.835182] LustreError: 324630:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11893.856926] LustreError: 324630:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [11893.952545] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11898.664857] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [11899.362358] LustreError: 324631:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11904.483320] LustreError: 324631:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11906.502858] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11906.753990] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11911.515837] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [11919.403522] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11926.294274] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11933.022872] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11933.366090] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11940.744797] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [11948.142738] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11955.403717] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11955.683336] Lustre: lustre-OST0001: new disk, initializing [11955.691818] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11955.699843] Lustre: 327467:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11955.841607] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11962.438890] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [11965.081037] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11965.090938] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11965.137126] LustreError: 327927:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11966.179446] LustreError: 327927:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11966.196184] LustreError: 327927:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11967.208206] LustreError: 327932:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11969.331450] LustreError: 327927:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11969.346445] LustreError: 327927:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 4 previous similar messages [11972.383928] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11972.597349] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11973.484443] LustreError: 327927:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11973.492084] LustreError: 327927:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [11978.196493] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9fae62f98000.ost_server_uuid 50 [11979.795715] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9fae62f98000.ost_server_uuid in FULL state after 0 sec [11981.673586] LustreError: 327932:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11981.686424] LustreError: 327932:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [11985.656360] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9fae62f98000.ost_server_uuid 50 [11987.402642] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9fae62f98000.ost_server_uuid in FULL state after 0 sec [11988.926730] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [12003.217960] Lustre: lustre-OST0001: Not available for connect from 192.168.201.31@tcp (stopping) [12004.322433] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12004.323648] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12004.330945] LustreError: Skipped 1 previous similar message [12004.343147] Lustre: Skipped 1 previous similar message [12008.299230] Lustre: lustre-OST0001: Not available for connect from 192.168.201.31@tcp (stopping) [12008.305136] Lustre: Skipped 4 previous similar messages [12009.077682] Lustre: server umount lustre-OST0001 complete [12011.497654] LustreError: 327933:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12011.513196] LustreError: 327933:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [12016.612707] LustreError: 326453:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12016.615865] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12016.616700] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12016.621825] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12016.633763] LustreError: 326453:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [12021.291824] Lustre: server umount lustre-OST0000 complete [12026.850534] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12026.870971] Lustre: Skipped 1 previous similar message [12031.201609] Lustre: server umount lustre-MDT0000 complete [12031.972214] LustreError: 328179:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12031.985604] LustreError: 328179:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [12034.919882] LustreError: 327465:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790908041 with bad export cookie 4558099093886209997 [12034.926760] LustreError: 327465:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12034.942604] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12035.466877] Lustre: server umount lustre-MDT0001 complete [12047.165713] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [12050.089365] Key type lgssc unregistered [12050.374977] LNet: 330157:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12050.383346] LNetError: 330157:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12050.397881] LNet: Removed LNI 192.168.201.131@tcp [12051.201568] Key type .llcrypt unregistered [12051.204073] Key type ._llcrypt unregistered [12078.192201] Key type ._llcrypt registered [12078.194064] Key type .llcrypt registered [12078.284969] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [12079.390504] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12079.533506] alg: No test for adler32 (adler32-zlib) [12080.704966] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [12080.922303] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [12082.577784] Key type lgssc registered [12083.940600] Lustre: Echo OBD driver; http://www.lustre.org/ [12092.533768] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12092.562797] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12094.151555] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12099.371410] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [12104.676230] LustreError: 331593:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12104.715519] LustreError: 331593:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [12107.869907] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12109.801194] LustreError: 331594:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12114.926814] LustreError: 331593:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12116.291720] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12116.780774] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12121.672992] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [12132.795635] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12133.201240] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12139.376565] LustreError: 333275:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12139.409774] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [12140.628753] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [12146.669760] LustreError: 333576:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12146.694928] LustreError: 333576:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [12150.791727] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12156.907992] LustreError: 333576:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12156.927794] LustreError: 333576:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [12159.990160] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12160.365739] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12165.645212] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [12167.979531] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [12177.151456] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12183.605656] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [12183.859155] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12293.605314] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12293.613961] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12293.645347] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12293.664289] Lustre: Skipped 1 previous similar message [12297.842816] Lustre: server umount lustre-OST0001 complete [12298.760112] LustreError: 333275:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12298.789744] LustreError: 333275:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [12305.379669] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12305.386033] LustreError: Skipped 1 previous similar message [12305.392098] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12305.404082] Lustre: Skipped 1 previous similar message [12305.412854] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12308.451300] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12308.464773] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12308.479207] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12314.086878] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12314.095925] Lustre: Skipped 2 previous similar messages [12319.201558] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12319.211478] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12319.221454] Lustre: Skipped 1 previous similar message [12319.344182] Lustre: server umount lustre-OST0000 complete [12326.882568] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12326.887957] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12329.450776] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12329.461033] Lustre: Skipped 2 previous similar messages [12331.206454] Lustre: server umount lustre-MDT0000 complete [12334.563786] LustreError: 332516:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12334.585061] LustreError: 332516:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 9 previous similar messages [12335.635249] LustreError: 331576:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790908341 with bad export cookie 7645204256066519668 [12335.636803] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12335.643045] LustreError: 331576:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12335.877278] Lustre: server umount lustre-MDT0001 complete [12348.724806] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [12352.235470] Key type lgssc unregistered [12352.607067] LNet: 336542:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12352.614796] LNetError: 336542:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12353.638962] LNet: Removed LNI 192.168.201.131@tcp [12354.645714] Key type .llcrypt unregistered [12354.647520] Key type ._llcrypt unregistered [12408.917232] Key type ._llcrypt registered [12408.919677] Key type .llcrypt registered [12409.032760] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [12410.085280] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12410.114191] alg: No test for adler32 (adler32-zlib) [12411.462430] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [12411.704915] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [12413.416511] Key type lgssc registered [12414.860798] Lustre: Echo OBD driver; http://www.lustre.org/ [12425.578285] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12425.599134] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12427.388252] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12433.073120] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [12436.968725] LustreError: 338847:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12436.980726] LustreError: 338847:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [12442.099294] LustreError: 338848:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12442.693980] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12443.170329] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12447.939918] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [12451.515603] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12460.235067] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12460.843857] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12468.020098] LustreError: 340337:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12470.570167] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [12472.191487] LustreError: 340338:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12472.213192] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12472.238427] LustreError: 340338:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [12477.430355] LustreError: 340764:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12477.480459] LustreError: 340764:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [12481.364545] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12481.630545] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12486.682659] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12488.803615] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [12499.069579] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12505.594816] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12519.907753] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12519.912252] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12519.925266] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12522.481325] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12522.492804] Lustre: Skipped 1 previous similar message [12522.495318] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12522.516220] Lustre: Skipped 3 previous similar messages [12525.762164] Lustre: server umount lustre-MDT0000 complete [12527.585326] LustreError: 338848:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12527.619577] LustreError: 338848:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [12529.756182] LustreError: 341915:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790908535 with bad export cookie 11632117808664992279 [12529.779055] LustreError: 341915:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12529.779244] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12530.023317] Lustre: server umount lustre-MDT0001 complete [12544.151612] Lustre: server umount lustre-OST0000 complete [12548.066395] Lustre: 338099:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790908538/real 1790908538] req@ffff8ab65952ce00 x1877903584333056/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790908554 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12548.091926] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12548.854521] Lustre: server umount lustre-OST0001 complete [12572.259433] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [12585.991829] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12586.510421] LustreError: 344114:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12586.597891] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12591.891380] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [12601.727350] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12602.148417] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12606.218836] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [12608.955852] Lustre: 345253:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12616.584125] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12623.558776] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [12627.256081] LustreError: 346139:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12627.275677] LustreError: 346139:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [12631.367749] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12632.698969] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12632.895189] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12632.903346] Lustre: Skipped 1 previous similar message [12638.199946] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12639.589366] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [12647.707773] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12652.291196] Lustre: 347129:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12658.657329] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12658.659572] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12658.677583] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12658.710670] Lustre: Skipped 3 previous similar messages [12663.783136] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12663.797815] Lustre: Skipped 6 previous similar messages [12664.498380] Lustre: server umount lustre-MDT0000 complete [12667.608911] LustreError: 344096:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790908673 with bad export cookie 11632117808664993980 [12667.629273] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12667.870284] Lustre: server umount lustre-MDT0001 complete [12682.444180] Lustre: server umount lustre-OST0000 complete [12685.287215] Lustre: 338100:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790908675/real 1790908675] req@ffff8ab781936d80 x1877903584383232/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790908691 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12685.322136] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12687.174228] Lustre: server umount lustre-OST0001 complete [12691.114914] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12693.630713] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12695.914405] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 22:38:19 (1790908699) [12726.339430] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [12737.010412] loop0: detected capacity change from 0 to 1099511627776 [12737.784484] SGI XFS with ACLs, security attributes, quota, no debug enabled [12737.804925] *** VALIDATE xfs *** [12737.846929] XFS (loop0): Mounting V5 Filesystem [12737.883954] XFS (loop0): Ending clean mount [12739.221422] loop1: detected capacity change from 0 to 18691697672192 [12875.493850] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_hostid [12890.246955] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [12937.021984] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12937.312565] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12937.391895] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12937.501310] Lustre: lustre-MDT0000: new disk, initializing [12937.581624] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12937.611639] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12942.738891] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [12954.011772] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12954.115673] Lustre: 352281:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12954.163669] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12954.167530] Lustre: Skipped 1 previous similar message [12954.332200] Lustre: lustre-MDT0001: new disk, initializing [12954.473622] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12954.491058] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12960.008660] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [12971.599157] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12978.033992] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12987.308822] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12987.795869] Lustre: lustre-OST0000: new disk, initializing [12987.810845] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12987.829022] Lustre: 354254:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12989.588363] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12989.615332] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12989.835047] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12995.151727] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [13006.406905] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13012.553836] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [13012.867224] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [13017.958658] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [13018.163475] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [13020.641459] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13020.644667] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13020.662240] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13020.672503] Lustre: Skipped 1 previous similar message [13026.938519] Lustre: server umount lustre-OST0000 complete [13036.005411] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13036.014608] Lustre: Skipped 3 previous similar messages [13046.240219] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13046.502335] Lustre: server umount lustre-MDT0000 complete [13050.250078] LustreError: 352274:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790909056 with bad export cookie 11632117808664995534 [13050.257476] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13050.261791] LustreError: 352274:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [13050.538813] Lustre: server umount lustre-MDT0001 complete [13060.246548] XFS (loop0): Unmounting Filesystem [13065.992777] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 22:44:29 (1790909069) [13073.587695] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13073.891063] LustreError: 356816:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13073.904794] LustreError: 356816:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [13073.955782] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13073.961236] Lustre: Skipped 2 previous similar messages [13078.683529] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [13087.104193] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13092.786282] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [13101.978285] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13108.833484] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13116.530339] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13123.463459] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [13131.772395] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13142.499369] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13142.504048] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13142.512847] Lustre: Skipped 3 previous similar messages [13142.528725] Lustre: Skipped 4 previous similar messages [13148.210340] Lustre: server umount lustre-OST0000 complete [13159.020380] Lustre: server umount lustre-MDT0000 complete [13163.299364] LustreError: 356798:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790909169 with bad export cookie 11632117808664996416 [13163.302135] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13163.316626] LustreError: 356798:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13163.701713] Lustre: server umount lustre-MDT0001 complete [13180.816490] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [13185.071593] Key type lgssc unregistered [13185.501596] LNet: 360459:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13185.526750] LNetError: 360459:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13186.604898] LNet: Removed LNI 192.168.201.131@tcp [13187.339198] Key type .llcrypt unregistered [13187.342225] Key type ._llcrypt unregistered [13220.501298] Key type ._llcrypt registered [13220.503156] Key type .llcrypt registered [13220.652384] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [13222.159196] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13222.269263] alg: No test for adler32 (adler32-zlib) [13223.602146] Lustre: Lustre: Build Version: 2.17.58_63_g3705aed [13223.909576] LNet: Added LNI 192.168.201.131@tcp [8/256/0/180] [13225.568959] Key type lgssc registered [13226.817540] Lustre: Echo OBD driver; http://www.lustre.org/ [13235.181182] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13235.192888] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13236.608905] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13242.555895] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [13245.925166] LustreError: 362044:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13245.958601] LustreError: 362044:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [13251.051679] LustreError: 362043:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13252.860571] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13253.174080] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13258.562805] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [13267.745725] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13276.284670] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13284.626585] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13284.892990] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13291.884641] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [13301.855811] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13319.592291] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.31@tcp (stopping) [13319.652103] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13319.656493] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13322.216246] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13322.219727] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13322.262823] Lustre: Skipped 2 previous similar messages [13324.020879] Lustre: server umount lustre-MDT0001 complete [13324.649678] LustreError: 362040:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.31@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13327.333149] LustreError: 362988:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13327.361295] LustreError: 362988:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [13332.469224] LustreError: 362043:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13332.491224] LustreError: 362043:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [13338.159213] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13338.227468] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13338.231357] Lustre: Found index 1 for lustre-MDT0001, updating log [13338.258205] Lustre: 362031:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13338.501137] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13343.748839] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13343.834772] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [13347.316747] LustreError: 365646:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13347.332860] LustreError: 365646:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13347.352232] LustreError: 362035:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13347.422915] LustreError: 365660:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.131@tcp: cfg command failed: rc = -17 [13347.429256] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [13347.429256] [13355.388871] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.31@tcp (stopping) [13359.081369] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13359.115740] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13361.391790] Lustre: server umount lustre-MDT0001 complete [13364.197298] LustreError: 362038:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13364.224337] LustreError: 362038:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [13374.019278] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13374.216616] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13374.224370] Lustre: Found index 1 for lustre-MDT0001, updating log [13374.318139] Lustre: 362033:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13374.720051] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13380.090051] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13381.079830] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [13384.211126] LustreError: 366713:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13384.222823] LustreError: 366713:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13384.249966] LustreError: 362035:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13384.253536] LustreError: 362035:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13384.352722] LustreError: 366721:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13384.373033] LustreError: 366721:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13384.392178] LustreError: 366721:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.131@tcp: cfg command failed: rc = -17 [13384.403950] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.131@tcp [13384.403950] [13688.241284] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.31@tcp (stopping) [13688.255808] Lustre: Skipped 2 previous similar messages [13692.387194] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13692.392231] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13692.438765] Lustre: Skipped 1 previous similar message [13693.973420] Lustre: server umount lustre-MDT0001 complete [13697.513632] LustreError: 367657:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13697.526984] LustreError: 367657:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [13706.875857] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13707.052290] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13707.058437] Lustre: Found index 1 for lustre-MDT0001, updating log [13707.097660] Lustre: 362031:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13707.414482] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13712.904973] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13713.418618] LustreError: 374565:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13713.431840] LustreError: 374565:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13713.445412] LustreError: 362035:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13713.462344] LustreError: 362035:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13713.596586] LustreError: 374571:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13713.621371] LustreError: 374571:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.131@tcp: cfg command failed: rc = -114 [13713.639328] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13713.639328] [13714.258140] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [14040.035862] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14040.050938] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14040.072550] Lustre: Skipped 1 previous similar message [14040.079767] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14040.089678] Lustre: Skipped 3 previous similar messages [14054.368213] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [14055.366854] Lustre: server umount lustre-MDT0000 complete [14055.904935] LustreError: 367657:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14055.921556] LustreError: 367657:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [14060.304761] LustreError: 363887:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790910066 with bad export cookie 14372224209967183922 [14060.321435] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14060.323633] LustreError: 363887:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14061.035692] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14061.050827] Lustre: Skipped 2 previous similar messages [14061.055370] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [14061.060304] Lustre: Skipped 8 previous similar messages [14067.142801] Lustre: server umount lustre-MDT0001 complete [14071.520334] Lustre: server umount lustre-OST0000 complete [14082.340072] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_hostid [14096.826895] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [14143.590640] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 23:02:27 (1790910147) [14151.556148] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14151.831713] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14151.858896] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14151.925822] Lustre: lustre-MDT0000: new disk, initializing [14152.000570] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14152.023323] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14157.237367] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [14168.904345] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14169.047170] Lustre: 384820:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14169.114968] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14169.118973] Lustre: Skipped 1 previous similar message [14169.192568] Lustre: lustre-MDT0001: new disk, initializing [14169.401431] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14169.474703] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14169.497408] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14175.017834] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [14187.211973] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14195.112779] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14204.603923] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14205.151535] Lustre: lustre-OST0000: new disk, initializing [14205.161973] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14205.178253] Lustre: 386791:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14205.378885] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14210.659082] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14210.677855] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14210.925429] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14213.977932] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_default_debug -1 all [14225.859640] Lustre: DEBUG MARKER: oleg131-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14235.617719] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [14235.631177] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14235.644869] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14235.647675] Lustre: Skipped 1 previous similar message [14241.315268] Lustre: server umount lustre-MDT0000 complete [14241.764818] LustreError: 384832:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14241.785092] LustreError: 384832:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [14245.186388] LustreError: 385627:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [14245.190086] LustreError: 386790:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790910251 with bad export cookie 14372224209967185875 [14245.197522] LustreError: MGC192.168.201.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14245.246497] LustreError: 386790:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [14245.562935] Lustre: server umount lustre-MDT0001 complete [14259.313602] Lustre: server umount lustre-OST0000 complete [14294.323764] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing set_hostid [14307.167386] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing load_modules_local [14354.971446] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 14091 sec ========= 23:05:58 (1790910358) [14357.277272] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 23:06:00 (1790910360) === [14361.812607] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 23:06:05 (1790910365) === [14407.641830] Lustre: DEBUG MARKER: oleg131-server.virtnet: executing unload_modules_local [14410.124429] Key type lgssc unregistered [14410.405447] LNet: 393165:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14410.413691] LNetError: 393165:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14411.449646] LNet: Removed LNI 192.168.201.131@tcp [14412.476654] Key type .llcrypt unregistered [14412.478976] Key type ._llcrypt unregistered