[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 504853460 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001013] APIC: Switch to symmetric I/O mode setup [ 0.002277] x2apic enabled [ 0.003011] Switched APIC routing to physical x2apic. [ 0.004016] kvm-guest: setup PV IPIs [ 0.006985] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007020] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008010] pid_max: default: 32768 minimum: 301 [ 0.009136] LSM: Security Framework initializing [ 0.010057] Yama: becoming mindful. [ 0.011044] SELinux: Initializing. [ 0.012077] *** VALIDATE selinux *** [ 0.020662] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.025290] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027157] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028112] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030111] *** VALIDATE tmpfs *** [ 0.031446] *** VALIDATE proc *** [ 0.032273] *** VALIDATE cgroup *** [ 0.033010] *** VALIDATE cgroup2 *** [ 0.034299] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036133] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038028] Spectre V2 : User space: Vulnerable [ 0.039009] Speculative Store Bypass: Vulnerable [ 0.041956] debug: unmapping init [mem 0xffffffffb9e59000-0xffffffffb9e60fff] [ 0.043162] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044698] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045026] ... version: 2 [ 0.046012] ... bit width: 48 [ 0.047010] ... generic registers: 4 [ 0.048011] ... value mask: 0000ffffffffffff [ 0.049016] ... max period: 00007fffffffffff [ 0.050011] ... fixed-purpose events: 3 [ 0.051011] ... event mask: 000000070000000f [ 0.052305] rcu: Hierarchical SRCU implementation. [ 0.054412] smp: Bringing up secondary CPUs ... [ 0.055613] x86: Booting SMP configuration: [ 0.056027] .... node #0, CPUs: #1 #2 #3 [ 0.063133] smp: Brought up 1 node, 4 CPUs [ 0.065010] smpboot: Max logical packages: 1 [ 0.066011] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.126581] node 0 deferred pages initialised in 58ms [ 0.129142] devtmpfs: initialized [ 0.130149] x86/mm: Memory block size: 128MB [ 0.132327] gcov: version magic: 0x41383552 [ 0.134178] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.135073] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.136224] pinctrl core: initialized pinctrl subsystem [ 0.137154] [ 0.137536] ************************************************************* [ 0.138011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.139015] ** ** [ 0.140009] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.141010] ** ** [ 0.142013] ** This means that this kernel is built to expose internal ** [ 0.143012] ** IOMMU data structures, which may compromise security on ** [ 0.144016] ** your system. ** [ 0.145012] ** ** [ 0.146012] ** If you see this message and you are not debugging the ** [ 0.147012] ** kernel, report this immediately to your vendor! ** [ 0.148012] ** ** [ 0.149013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.150008] ************************************************************* [ 0.151733] NET: Registered protocol family 16 [ 0.152448] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.153057] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.154068] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.155615] cpuidle: using governor menu [ 0.157349] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.159526] PCI: Using configuration type 1 for base access [ 0.162187] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.171152] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.172030] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.173242] cryptd: max_cpu_qlen set to 1000 [ 0.175016] ACPI: Added _OSI(Module Device) [ 0.176015] ACPI: Added _OSI(Processor Device) [ 0.177012] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.178013] ACPI: Added _OSI(Processor Aggregator Device) [ 0.182257] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.185242] ACPI: Interpreter enabled [ 0.186064] ACPI: PM: (supports S0 S3 S4 S5) [ 0.187015] ACPI: Using IOAPIC for interrupt routing [ 0.188134] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.189412] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.199012] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.200046] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.201018] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.202078] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.204418] acpiphp: Slot [2] registered [ 0.205115] acpiphp: Slot [5] registered [ 0.206139] acpiphp: Slot [6] registered [ 0.207131] acpiphp: Slot [7] registered [ 0.208087] acpiphp: Slot [8] registered [ 0.209083] acpiphp: Slot [9] registered [ 0.210114] acpiphp: Slot [10] registered [ 0.211163] acpiphp: Slot [3] registered [ 0.212111] acpiphp: Slot [4] registered [ 0.213101] acpiphp: Slot [11] registered [ 0.214120] acpiphp: Slot [12] registered [ 0.215110] acpiphp: Slot [13] registered [ 0.216105] acpiphp: Slot [14] registered [ 0.217118] acpiphp: Slot [15] registered [ 0.218153] acpiphp: Slot [16] registered [ 0.219112] acpiphp: Slot [17] registered [ 0.220114] acpiphp: Slot [18] registered [ 0.221094] acpiphp: Slot [19] registered [ 0.222110] acpiphp: Slot [20] registered [ 0.223093] acpiphp: Slot [21] registered [ 0.224082] acpiphp: Slot [22] registered [ 0.225125] acpiphp: Slot [23] registered [ 0.226093] acpiphp: Slot [24] registered [ 0.227087] acpiphp: Slot [25] registered [ 0.228124] acpiphp: Slot [26] registered [ 0.229085] acpiphp: Slot [27] registered [ 0.230081] acpiphp: Slot [28] registered [ 0.231118] acpiphp: Slot [29] registered [ 0.232130] acpiphp: Slot [30] registered [ 0.233029] acpiphp: Slot [31] registered [ 0.235088] PCI host bridge to bus 0000:00 [ 0.236016] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.238022] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.240026] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.243025] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.246035] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.249027] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.251198] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.254164] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.258425] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.266015] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.270085] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.272012] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.273010] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.274010] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.276421] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.278520] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.279030] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.282397] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.289019] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.302014] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.307016] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.311709] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.324022] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.334021] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.361021] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.373574] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.382019] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.394016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.419023] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.431126] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.441017] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.448029] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.461019] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.471189] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.483017] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.499025] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.559015] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.589169] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.602029] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.607026] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.622022] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.635881] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.659019] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.679016] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.732023] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.760441] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.763471] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.765462] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.769498] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.771309] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.777261] iommu: Default domain type: Passthrough [ 0.778490] SCSI subsystem initialized [ 0.779207] ACPI: bus type USB registered [ 0.780100] usbcore: registered new interface driver usbfs [ 0.781090] usbcore: registered new interface driver hub [ 0.782061] usbcore: registered new device driver usb [ 0.783134] pps_core: LinuxPPS API ver. 1 registered [ 0.784009] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.786056] PTP clock support registered [ 0.788039] EDAC MC: Ver: 3.0.0 [ 0.789539] PCI: Using ACPI for IRQ routing [ 0.792030] NetLabel: Initializing [ 0.793014] NetLabel: domain hash size = 128 [ 0.794014] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.796105] NetLabel: unlabeled traffic allowed by default [ 0.798135] vgaarb: loaded [ 0.799375] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.800012] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.805727] clocksource: Switched to clocksource kvm-clock [ 0.910574] VFS: Disk quotas dquot_6.6.0 [ 0.912120] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.914307] *** VALIDATE ramfs *** [ 0.915126] *** VALIDATE hugetlbfs *** [ 0.916784] pnp: PnP ACPI init [ 0.918432] pnp: PnP ACPI: found 6 devices [ 0.933982] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.936351] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.938505] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.940023] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.941542] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.943218] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.945128] NET: Registered protocol family 2 [ 0.946809] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.950612] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.954626] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.959115] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.961882] TCP: Hash tables configured (established 65536 bind 65536) [ 0.964527] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.968992] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.972512] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.976247] NET: Registered protocol family 1 [ 0.979430] RPC: Registered named UNIX socket transport module. [ 0.982103] RPC: Registered udp transport module. [ 0.983981] RPC: Registered tcp transport module. [ 0.985735] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.988314] NET: Registered protocol family 44 [ 0.990182] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.992054] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.994032] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.996323] PCI: CLS 0 bytes, default 64 [ 0.998374] Unpacking initramfs... [ 2.447583] debug: unmapping init [mem 0xffff8c123cc54000-0xffff8c123ffbffff] [ 2.451538] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.453827] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.456931] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.945138] Initialise system trusted keyrings [ 2.946711] Key type blacklist registered [ 2.948300] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.956934] zbud: loaded [ 2.960101] *** VALIDATE nfs *** [ 2.961132] *** VALIDATE nfs4 *** [ 2.962590] pstore: using deflate compression [ 2.965391] Platform Keyring initialized [ 3.067696] NET: Registered protocol family 38 [ 3.069186] Key type asymmetric registered [ 3.070408] Asymmetric key parser 'x509' registered [ 3.071922] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.074519] io scheduler mq-deadline registered [ 3.075843] io scheduler kyber registered [ 3.077208] io scheduler bfq registered [ 3.078881] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.081578] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.083987] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.086493] ACPI: Power Button [PWRF] [ 3.094485] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.102321] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.116196] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.125517] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.146377] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.175589] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.206412] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.211213] Non-volatile memory driver v1.3 [ 3.212982] Linux agpgart interface v0.103 [ 3.249849] virtio_blk virtio1: [vda] 150648 512-byte logical blocks (77.1 MB/73.6 MiB) [ 3.253320] vda: detected capacity change from 0 to 77131776 [ 3.273866] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.276847] vdb: detected capacity change from 0 to 1073741824 [ 3.294123] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.296971] vdc: detected capacity change from 0 to 2621440000 [ 3.313629] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.316257] vdd: detected capacity change from 0 to 2621440000 [ 3.334036] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.336379] vde: detected capacity change from 0 to 4294967296 [ 3.350755] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.353713] vdf: detected capacity change from 0 to 4294967296 [ 3.362465] libphy: Fixed MDIO Bus: probed [ 3.369324] usbcore: registered new interface driver usbserial_generic [ 3.371964] usbserial: USB Serial support registered for generic [ 3.374967] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.380155] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.382577] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.385836] mousedev: PS/2 mouse device common for all mice [ 3.389707] rtc_cmos 00:05: RTC can wake from S4 [ 3.393382] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.397089] rtc_cmos 00:05: registered as rtc0 [ 3.399905] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.402889] intel_pstate: CPU model not supported [ 3.406054] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.407648] hid: raw HID events driver (C) Jiri Kosina [ 3.412257] usbcore: registered new interface driver usbhid [ 3.414413] usbhid: USB HID core driver [ 3.414722] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.415949] drop_monitor: Initializing network drop monitor service [ 3.421668] Initializing XFRM netlink socket [ 3.423849] NET: Registered protocol family 10 [ 3.427069] Segment Routing with IPv6 [ 3.428679] NET: Registered protocol family 17 [ 3.430968] mpls_gso: MPLS GSO support [ 3.436871] RAS: Correctable Errors collector initialized. [ 3.439239] AVX version of gcm_enc/dec engaged. [ 3.441413] AES CTR mode by8 optimization enabled [ 3.527494] sched_clock: Marking stable (3527463627, 0)->(4504449104, -976985477) [ 3.531675] registered taskstats version 1 [ 3.533803] Loading compiled-in X.509 certificates [ 3.536035] zswap: loaded using pool lzo/zbud [ 3.565747] Key type big_key registered [ 3.592341] Key type encrypted registered [ 3.593664] ima: No TPM chip found, activating TPM-bypass! [ 3.595435] ima: Allocated hash algorithm: sha1 [ 3.597098] ima: No architecture policies found [ 3.599093] evm: Initialising EVM extended attributes: [ 3.601258] evm: security.selinux [ 3.602581] evm: security.ima [ 3.603804] evm: security.capability [ 3.605449] evm: HMAC attrs: 0x1 [ 3.608478] rtc_cmos 00:05: setting system clock to 2026-09-30 02:12:28 UTC (1790734348) [ 3.615349] debug: unmapping init [mem 0xffffffffbae03000-0xffffffffbaffffff] [ 3.618341] debug: unmapping init [mem 0xffffffffb9b82000-0xffffffffb9e58fff] [ 3.627178] Write protecting the kernel read-only data: 28672k [ 3.630639] debug: unmapping init [mem 0xffffffffb8203000-0xffffffffb83fffff] [ 3.633551] debug: unmapping init [mem 0xffffffffb8b14000-0xffffffffb8bfffff] [ 3.671539] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.680268] systemd[1]: Detected virtualization kvm. [ 3.682021] systemd[1]: Detected architecture x86-64. [ 3.683424] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.710303] systemd[1]: No hostname configured. [ 3.712208] systemd[1]: Set hostname to . [ 3.714129] random: systemd: uninitialized urandom read (16 bytes read) [ 3.716885] systemd[1]: Initializing machine ID from random generator. [ 3.874255] random: systemd: uninitialized urandom read (16 bytes read) [ 3.877081] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.880494] random: systemd: uninitialized urandom read (16 bytes read) [ 3.883464] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 3.889080] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Initrd Root Device. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Swap. Starting Apply Kernel Variables... Starting Setup Virtual Console... [ OK ] Reached target Timers. [ OK ] Reached target Paths. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.591428] device-mapper: uevent: version 1.0.3 [ 4.594396] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.537695] random: fast init done [ 5.593699] scsi host0: ata_piix [ 5.629145] virtio_net virtio0 ens2: renamed from eth0 [ 5.644221] scsi host1: ata_piix [ 5.645959] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.648088] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.411353] dracut-initqueue[579]: RTNETLINK answers: File exists [ 10.110861] random: crng init done [ 10.112544] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.820047] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Swap. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.111035] printk: systemd: 25 output lines suppressed due to ratelimiting [ 12.364711] SELinux: Disabled at runtime. [ 12.428901] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.438061] systemd[1]: Detected virtualization kvm. [ 12.440056] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.953666] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.958091] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.964826] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.972282] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.975945] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.983437] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.990720] systemd[1]: Mounting Kernel Debug File System... Mounting Kernel Debug File System... [ OK ] Created slice User and Session Slice. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Listening on udev Control Socket. Mounting POSIX Message Queue File System... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on Process Core Dump Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Activating swap /dev/disk/by-label/SWAP... Starting udev Coldplug all Devices... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. Mounting Huge Pages File System... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ 13.093682] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Reached target Slices. [ OK ] Reached target rpc_pipefs.target. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target RPC Port Mapper. Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... Starting Remount Root and Kernel File Systems... [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-getty.slice. [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 13.461616] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.818101] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.847028] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.937512] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.948593] EDAC sbridge: Ver: 1.1.2 [ 15.446184] Key type dns_resolver registered [ 15.766422] NFS: Registering the id_resolver key type [ 15.768894] Key type id_resolver registered [ 15.770679] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started irqbalance daemon. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Reached target sshd-keygen.target. Starting Restore /run/initramfs on shutdown... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg145-server login: [ 60.651889] libcfs: loading out-of-tree module taints kernel. [ 60.741963] Key type ._llcrypt registered [ 60.745602] Key type .llcrypt registered [ 60.835740] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_hostid [ 78.977833] hrtimer: interrupt took 6819806 ns [ 82.003440] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 84.043790] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 84.066143] alg: No test for adler32 (adler32-zlib) [ 85.645516] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [ 86.777249] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [ 88.648237] Key type lgssc registered [ 90.791722] Lustre: Echo OBD driver; http://www.lustre.org/ [ 112.267373] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 154.742930] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 169.885342] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 169.941242] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 171.473815] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 171.528643] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 171.659855] Lustre: lustre-MDT0000: new disk, initializing [ 171.758685] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 171.781671] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 177.368488] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 191.769088] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 191.905949] Lustre: 6509:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 191.951232] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 191.960891] Lustre: Skipped 1 previous similar message [ 192.062990] Lustre: lustre-MDT0001: new disk, initializing [ 192.143190] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 192.163623] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 192.175455] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 197.679343] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 202.788789] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 214.052175] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 214.439336] Lustre: lustre-OST0000: new disk, initializing [ 214.447598] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 214.455828] Lustre: 8446:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 214.533529] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 215.912639] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 215.920356] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 216.000865] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 221.569483] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 238.237706] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 238.405710] Lustre: lustre-OST0001: new disk, initializing [ 238.410255] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 238.415942] Lustre: 9518:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 238.498000] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 245.325456] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 245.346776] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 245.441660] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 245.712808] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 258.935798] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 264.025294] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 270.723336] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing check_logdir /tmp/testlogs/ [ 275.922354] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing yml_node [ 280.374800] Lustre: DEBUG MARKER: Client: 2.17.59.3 [ 282.929680] Lustre: DEBUG MARKER: MDS: 2.17.59.3 [ 285.640359] Lustre: DEBUG MARKER: OSS: 2.17.59.3 [ 287.740329] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Sep 29 22:17:10 EDT 2026 [ 308.059536] Lustre: DEBUG MARKER: only running tests: 76-120 [ 309.966795] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 311.943615] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 330.724055] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 330.736578] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 330.762359] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 332.776339] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 332.786240] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 332.794323] Lustre: Skipped 1 previous similar message [ 332.799548] Lustre: Skipped 1 previous similar message [ 335.311645] Lustre: server umount lustre-MDT0000 complete [ 339.873044] LustreError: 6500:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790734684 with bad export cookie 5787273549968661007 [ 339.875922] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 339.885124] LustreError: 6500:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 343.009499] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 343.022028] LustreError: 6516:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 343.024322] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 343.024328] Lustre: Skipped 2 previous similar messages [ 343.046917] Lustre: Skipped 2 previous similar messages [ 343.072576] LustreError: 6516:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [ 344.034433] LustreError: 6514:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 344.058683] LustreError: 6514:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 349.160592] LustreError: 6516:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 349.160905] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 349.192203] Lustre: Skipped 4 previous similar messages [ 354.273337] LustreError: 6514:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 354.274085] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 354.274360] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 4. Is it stuck? [ 354.296387] LustreError: 6514:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 354.328643] Lustre: Skipped 1 previous similar message [ 354.635887] Lustre: server umount lustre-MDT0001 complete [ 358.852624] Lustre: server umount lustre-OST0000 complete [ 362.418546] Lustre: server umount lustre-OST0001 complete [ 369.143703] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_hostid [ 379.112121] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 416.358734] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 416.616877] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 416.651133] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 416.706482] Lustre: lustre-MDT0000: new disk, initializing [ 416.779396] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 416.795150] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 421.852616] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 433.082050] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 433.288819] Lustre: 15122:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 433.423805] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 433.428386] Lustre: Skipped 1 previous similar message [ 433.511658] Lustre: lustre-MDT0001: new disk, initializing [ 433.686695] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 433.736614] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 433.755891] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 439.425659] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 451.696474] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 457.837990] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 464.434726] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 464.864705] Lustre: lustre-OST0000: new disk, initializing [ 464.868916] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 464.875587] Lustre: 17092:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 464.963797] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 466.060970] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 466.071185] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 466.239652] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 471.948670] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 481.967885] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 487.066579] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 487.417644] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 493.793761] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 494.121063] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 497.126390] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 497.127522] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 497.131941] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 497.148596] LustreError: Skipped 1 previous similar message [ 497.174363] Lustre: Skipped 1 previous similar message [ 503.007272] Lustre: server umount lustre-OST0000 complete [ 510.436885] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 510.453204] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 514.382720] Lustre: server umount lustre-MDT0000 complete [ 517.603023] LustreError: 15128:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 517.628727] LustreError: 15128:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 517.927362] LustreError: 15115:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790734862 with bad export cookie 5787273549968662540 [ 517.933275] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 517.936266] LustreError: 15115:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 518.257889] Lustre: server umount lustre-MDT0001 complete [ 522.419614] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 22:21:05 (1790734865) [ 528.608298] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 528.904808] LustreError: 19356:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 528.966451] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 532.402199] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 540.525233] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 546.169647] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 553.009272] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 558.751207] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 565.241193] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 572.415702] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 581.644981] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 586.217973] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 594.402809] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 594.404846] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 594.447939] Lustre: Skipped 3 previous similar messages [ 594.466806] Lustre: Skipped 8 previous similar messages [ 600.291216] Lustre: server umount lustre-MDT0000 complete [ 604.313974] LustreError: 21199:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790734949 with bad export cookie 5787273549968663422 [ 604.321572] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 604.337119] LustreError: 21199:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 604.690867] Lustre: server umount lustre-MDT0001 complete [ 619.584232] Lustre: server umount lustre-OST0000 complete [ 640.007814] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 650.898298] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 651.359868] LustreError: 23738:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 651.385946] LustreError: 23738:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 651.475645] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 651.479156] Lustre: Skipped 2 previous similar messages [ 656.286986] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 667.405442] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 674.143914] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 677.490821] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 684.368298] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 693.503162] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 701.542707] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 701.737879] Lustre: lustre-OST0001: new disk, initializing [ 701.744540] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 701.753371] Lustre: 26058:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 703.126613] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 703.142646] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 703.271805] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 708.061164] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 717.924312] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 721.509213] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 734.182793] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 734.186423] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 734.198592] Lustre: Skipped 3 previous similar messages [ 734.216659] Lustre: Skipped 6 previous similar messages [ 740.219444] Lustre: server umount lustre-MDT0000 complete [ 744.055517] LustreError: 23719:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790735088 with bad export cookie 5787273549968664269 [ 744.060488] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 744.070830] LustreError: 23719:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 744.408917] Lustre: server umount lustre-MDT0001 complete [ 758.392921] Lustre: server umount lustre-OST0000 complete [ 760.291832] Lustre: 3638:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790735089/real 1790735089] req@ffff8c11828a3800 x1877721146973440/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790735105 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 762.440614] Lustre: server umount lustre-OST0001 complete [ 781.876460] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 793.176029] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 793.554948] LustreError: 29078:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 793.572888] LustreError: 29078:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 793.654398] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 793.661967] Lustre: Skipped 3 previous similar messages [ 798.628351] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 807.537809] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 812.494038] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 815.716613] Lustre: 30225:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 821.765814] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 828.130938] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 836.684904] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 844.429314] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 853.272202] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 858.430667] Lustre: 32103:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 869.346064] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 869.354649] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 869.378207] Lustre: Skipped 1 previous similar message [ 869.384743] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 869.390404] Lustre: Skipped 4 previous similar messages [ 872.877726] Lustre: server umount lustre-MDT0000 complete [ 874.983508] LustreError: 29813:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 875.025947] LustreError: 29813:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [ 876.860408] LustreError: 29058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790735221 with bad export cookie 5787273549968665564 [ 876.874596] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 877.338157] Lustre: server umount lustre-MDT0001 complete [ 891.763560] Lustre: server umount lustre-OST0000 complete [ 895.328946] Lustre: 3640:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790735224/real 1790735224] req@ffff8c12871e5880 x1877721147024512/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790735240 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 895.613792] Lustre: server umount lustre-OST0001 complete [ 905.282252] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 22:27:28 (1790735248) [ 931.172894] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 940.532626] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 945.267304] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 954.506074] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 959.826305] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 962.837592] Lustre: 36054:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 972.171914] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 980.424865] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 989.421431] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 996.229243] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 1004.104886] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1007.908889] Lustre: 37937:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1015.285445] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1015.294675] Lustre: Skipped 6 previous similar messages [ 1015.301272] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1015.305796] Lustre: Skipped 5 previous similar messages [ 1019.459697] Lustre: server umount lustre-MDT0000 complete [ 1020.394785] LustreError: 34906:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1020.422191] LustreError: 34906:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 10 previous similar messages [ 1024.243921] LustreError: 36057:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790735369 with bad export cookie 5787273549968666670 [ 1024.245639] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1024.260094] LustreError: 36057:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 1024.658858] Lustre: server umount lustre-MDT0001 complete [ 1039.463912] Lustre: server umount lustre-OST0000 complete [ 1041.696107] Lustre: 3638:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790735370/real 1790735370] req@ffff8c1183b42a00 x1877721147076096/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790735386 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1044.460312] Lustre: server umount lustre-OST0001 complete [ 1058.166583] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 22:29:59 (1790735399) [ 1076.077672] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 1086.537506] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1087.035151] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1087.040257] Lustre: Skipped 7 previous similar messages [ 1092.071899] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 1100.753704] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1105.694784] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 1108.438426] Lustre: 41227:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1115.776645] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1121.709882] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 1130.592736] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1137.352630] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 1144.315505] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1148.064368] Lustre: 43106:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1149.037342] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1149.048111] Lustre: Skipped 1 previous similar message [ 1152.488069] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1152.488917] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1152.515660] Lustre: Skipped 3 previous similar messages [ 1158.344032] Lustre: server umount lustre-MDT0000 complete [ 1162.094149] LustreError: 41611:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790735506 with bad export cookie 5787273549968667769 [ 1162.098101] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1162.113156] LustreError: 41611:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1168.811335] Lustre: server umount lustre-MDT0001 complete [ 1176.547466] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1187.297682] LustreError: 3637:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8c1291769880 x1877721147126016/t0(0) o250->MGC192.168.201.145@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1192.964681] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 1198.075321] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1203.605066] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1208.790847] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 1209.346037] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1209.361042] Lustre: Skipped 1 previous similar message [ 1216.943787] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1218.764765] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1224.884138] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1226.746062] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1234.403507] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1241.282825] Lustre: server umount lustre-MDT0000 complete [ 1245.077214] LustreError: 40060:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790735589 with bad export cookie 5787273549968668875 [ 1245.080698] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1245.087540] LustreError: 40060:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1245.350892] Lustre: server umount lustre-MDT0001 complete [ 1255.021923] Lustre: server umount lustre-OST0000 complete [ 1258.905317] Lustre: server umount lustre-OST0001 complete [ 1269.465996] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 22:33:31 (1790735611) [ 1292.014894] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 1303.094534] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1303.543495] LustreError: 47728:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1303.571538] LustreError: 47728:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 27 previous similar messages [ 1307.571428] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 1315.579850] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1320.478103] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 1323.140173] Lustre: 48881:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1330.294576] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1338.584261] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 1348.474262] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1355.548824] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 1363.837884] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1369.767399] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1390.065929] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1390.094415] Lustre: Skipped 23 previous similar messages [ 1394.829814] Lustre: server umount lustre-MDT0000 complete [ 1400.240543] LustreError: 47709:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790735745 with bad export cookie 5787273549968669694 [ 1400.251224] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1400.549897] Lustre: server umount lustre-MDT0001 complete [ 1407.973709] Lustre: server umount lustre-OST0000 complete [ 1412.789558] Lustre: server umount lustre-OST0001 complete [ 1423.405696] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 22:36:06 (1790735766) [ 1425.577469] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1427.371611] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 22:36:10 (1790735770) [ 1437.742460] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_hostid [ 1447.950654] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 1470.974345] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1471.255843] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1471.381722] Lustre: lustre-MDT0000: new disk, initializing [ 1471.493241] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1476.118429] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 1487.443927] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1494.743521] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1495.038255] Lustre: lustre-OST0000: new disk, initializing [ 1495.044262] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1495.048899] Lustre: Skipped 1 previous similar message [ 1495.054928] Lustre: 54978:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1497.080189] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1497.100352] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1497.133412] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1501.954063] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 1510.550499] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1512.567835] Lustre: DEBUG MARKER: create test files [ 1548.262221] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1548.274446] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1548.293432] Lustre: Skipped 16 previous similar messages [ 1553.046322] Lustre: server umount lustre-OST0000 complete [ 1557.083731] Lustre: server umount lustre-MDT0000 complete [ 1569.627541] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [ 1572.661291] Key type lgssc unregistered [ 1572.951982] LNet: 56605:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1572.958247] LNetError: 56605:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1572.977892] LNet: Removed LNI 192.168.201.145@tcp [ 1574.047272] Key type .llcrypt unregistered [ 1574.049267] Key type ._llcrypt unregistered [ 1601.758367] Key type ._llcrypt registered [ 1601.760348] Key type .llcrypt registered [ 1601.831971] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 1602.488560] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1602.674739] alg: No test for adler32 (adler32-zlib) [ 1603.718958] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [ 1603.929963] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [ 1605.576237] Key type lgssc registered [ 1606.696388] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1614.058549] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1614.083622] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1615.473436] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1619.854588] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 1627.049877] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1635.134979] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1635.497341] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1640.645378] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1640.717347] Lustre: 59351:0:(osd_internal.h:1462:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1640.730369] Lustre: 59351:0:(osd_handler.c:2097:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1640.749230] Lustre: 59351:0:(osd_handler.c:2104:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1640.764183] Lustre: 59351:0:(osd_handler.c:2114:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1640.777340] Lustre: 59351:0:(osd_handler.c:2121:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1640.788935] Lustre: 59351:0:(osd_handler.c:2128:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1641.719768] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 1648.794284] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1654.190218] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1660.760713] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1670.343672] Lustre: server umount lustre-OST0000 complete [ 1685.825139] Lustre: server umount lustre-MDT0000 complete [ 1696.943816] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [ 1700.796419] Key type lgssc unregistered [ 1701.133564] LNet: 60958:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1701.148608] LNetError: 60958:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1701.165910] LNet: Removed LNI 192.168.201.145@tcp [ 1702.169151] Key type .llcrypt unregistered [ 1702.170876] Key type ._llcrypt unregistered [ 1736.138062] Key type ._llcrypt registered [ 1736.140298] Key type .llcrypt registered [ 1736.209626] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 1737.518065] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1737.585742] alg: No test for adler32 (adler32-zlib) [ 1738.873445] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [ 1739.280600] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [ 1740.985078] Key type lgssc registered [ 1742.747925] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1751.256137] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1751.297532] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1752.734612] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1757.472541] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 1765.683575] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1772.167146] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1772.503613] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1777.654865] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1777.774904] Lustre: 63808:0:(osd_internal.h:1462:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1777.792782] Lustre: 63808:0:(osd_handler.c:2097:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1777.798251] Lustre: 63808:0:(osd_handler.c:2104:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1777.803792] Lustre: 63808:0:(osd_handler.c:2114:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1777.812875] Lustre: 63808:0:(osd_handler.c:2121:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1777.818963] Lustre: 63808:0:(osd_handler.c:2128:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1778.673410] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 1787.730495] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1792.926666] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1803.235635] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1803.252502] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1803.268711] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1807.251397] Lustre: server umount lustre-OST0000 complete [ 1811.288791] Lustre: server umount lustre-MDT0000 complete [ 1821.563942] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [ 1824.293915] Key type lgssc unregistered [ 1824.599797] LNet: 65363:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1824.610230] LNetError: 65363:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1824.622799] LNet: Removed LNI 192.168.201.145@tcp [ 1825.510650] Key type .llcrypt unregistered [ 1825.512889] Key type ._llcrypt unregistered [ 1845.525675] Key type ._llcrypt registered [ 1845.527465] Key type .llcrypt registered [ 1845.647487] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_hostid [ 1864.612813] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 1866.230288] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1866.287061] alg: No test for adler32 (adler32-zlib) [ 1867.655820] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [ 1868.164727] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [ 1869.930152] Key type lgssc registered [ 1872.105898] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1909.279118] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1909.320912] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1910.591630] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1910.624148] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1910.721511] Lustre: lustre-MDT0000: new disk, initializing [ 1910.841906] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1910.867561] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1915.057340] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 1926.596590] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1926.795254] Lustre: 68534:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1926.901816] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1926.912483] Lustre: Skipped 1 previous similar message [ 1927.034751] Lustre: lustre-MDT0001: new disk, initializing [ 1927.236593] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1927.294409] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1927.323363] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1932.056389] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 1941.150444] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1946.859571] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1953.517037] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1953.856580] Lustre: lustre-OST0000: new disk, initializing [ 1953.860475] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1953.865599] Lustre: 70502:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1953.955568] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1960.054334] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1960.061854] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1960.240433] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1960.815728] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 1970.634705] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1974.680126] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1974.901423] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1979.540359] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1979.810827] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1986.020961] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1986.029701] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1986.030151] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1986.041313] Lustre: Skipped 1 previous similar message [ 1986.063537] Lustre: Skipped 1 previous similar message [ 1987.743432] Lustre: server umount lustre-OST0000 complete [ 1993.705048] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1993.713932] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1993.734296] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1996.262748] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1996.263780] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1997.697584] Lustre: server umount lustre-MDT0000 complete [ 2001.802138] LustreError: 68525:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790736346 with bad export cookie 5767902620884598934 [ 2001.806053] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2001.811317] LustreError: 68525:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2002.189417] Lustre: server umount lustre-MDT0001 complete [ 2011.478227] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 22:45:54 (1790736354) [ 2022.768607] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 2062.867610] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_hostid [ 2072.676388] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 2117.029184] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2117.415216] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2117.451179] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2117.532261] Lustre: lustre-MDT0000: new disk, initializing [ 2117.655074] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2117.678195] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2122.796438] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 2135.799431] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2136.027785] Lustre: 76523:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2136.124243] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2136.129588] Lustre: Skipped 1 previous similar message [ 2136.238813] Lustre: lustre-MDT0001: new disk, initializing [ 2136.335818] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2136.390373] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2136.401706] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2140.927569] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 2151.152392] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2157.170181] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2165.434405] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2165.986714] Lustre: lustre-OST0000: new disk, initializing [ 2165.990314] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2166.001196] Lustre: 78496:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2166.161637] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2172.582812] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2172.589892] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2172.661823] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2174.611816] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 2187.077461] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2191.481941] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2191.751411] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2198.353253] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2198.693656] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2203.617348] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2203.638172] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2203.663644] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2203.666358] Lustre: Skipped 1 previous similar message [ 2208.738229] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2208.749158] Lustre: Skipped 1 previous similar message [ 2215.393187] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2215.808439] Lustre: server umount lustre-OST0000 complete [ 2223.603205] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2223.613917] LustreError: Skipped 1 previous similar message [ 2223.615676] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2223.626238] Lustre: Skipped 1 previous similar message [ 2223.630442] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2223.634650] Lustre: Skipped 2 previous similar messages [ 2227.055297] Lustre: server umount lustre-MDT0000 complete [ 2229.219334] LustreError: 76535:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2229.251506] LustreError: 76535:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [ 2231.345956] LustreError: 76516:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790736576 with bad export cookie 5767902620884599816 [ 2231.353859] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2231.361748] LustreError: 76516:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2231.697879] Lustre: server umount lustre-MDT0001 complete [ 2240.430366] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 22:49:43 (1790736583) [ 2247.572132] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2248.033616] LustreError: 80910:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2248.108436] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2252.406663] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 2253.289755] LustreError: 80911:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2257.379269] LustreError: 80910:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2260.966546] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2266.026447] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 2273.168320] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2278.764354] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2286.316025] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2286.704442] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2286.718723] Lustre: Skipped 1 previous similar message [ 2293.618782] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 2302.257992] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2306.320712] Lustre: 83584:0:(genops.c:1773:obd_export_evict_by_uuid()) MGS: evicting 52a529c7-9064-4e3a-be52-b0636b2371e8 at adminstrative request [ 2308.581329] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2308.598828] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2318.837585] Lustre: MGS: Client 52a529c7-9064-4e3a-be52-b0636b2371e8 (at 0@lo) reconnecting [ 2318.864870] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x500bb4dfc0df077a to 0x500bb4dfc0df0a1a [ 2318.888456] Lustre: MGC192.168.201.145@tcp: Connection restored to 0@lo (at 0@lo) [ 2346.291360] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2346.499479] Lustre: lustre-OST0001: new disk, initializing [ 2346.502542] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2346.510464] Lustre: 83977:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2354.875411] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 2356.368243] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2356.386291] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2356.502206] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2364.782291] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2372.070225] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2372.074083] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2372.104540] Lustre: Skipped 1 previous similar message [ 2372.113674] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2372.144915] Lustre: Skipped 3 previous similar messages [ 2382.306427] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2382.606785] Lustre: server umount lustre-OST0001 complete [ 2387.430685] LustreError: 83597:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2387.453813] LustreError: 83597:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2397.678946] LustreError: 83597:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2397.706766] LustreError: 83597:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 2401.248111] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2401.508247] Lustre: server umount lustre-OST0000 complete [ 2407.906181] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2407.918938] Lustre: Skipped 3 previous similar messages [ 2407.925120] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2407.939243] Lustre: Skipped 11 previous similar messages [ 2411.699053] Lustre: server umount lustre-MDT0000 complete [ 2415.649578] LustreError: 80892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790736760 with bad export cookie 5767902620884601370 [ 2415.662019] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2415.992852] Lustre: server umount lustre-MDT0001 complete [ 2424.770476] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 22:52:47 (1790736767) [ 2426.604977] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2428.465897] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 22:52:51 (1790736771) [ 2430.487849] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2432.834915] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 22:52:55 (1790736775) [ 2435.066260] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2436.930681] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 22:52:59 (1790736779) [ 2438.616944] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2441.015834] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 22:53:03 (1790736783) [ 2443.379653] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2450.466988] loop: module loaded [ 2450.523262] loop0: detected capacity change from 0 to 4294967296 [ 2450.607282] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2452.260501] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2456.039503] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2460.231230] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2491.644039] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2491.666217] LustreError: 88104:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2491.670918] LustreError: 88104:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2491.679889] LustreError: 88104:0:(tgt_mount.c:2581:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2491.689397] LustreError: 88104:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2512.183871] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_hostid [ 2521.059119] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 2557.148897] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2557.407894] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2557.435238] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2557.488674] Lustre: lustre-MDT0000: new disk, initializing [ 2557.543609] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2557.548153] Lustre: Skipped 1 previous similar message [ 2557.561715] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2563.207307] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 2573.402412] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2573.563814] Lustre: 90972:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2573.923368] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2578.359969] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 2587.619350] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2594.052562] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2600.701337] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2600.964314] Lustre: 92940:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2607.181746] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2607.317808] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2608.593661] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 2618.340885] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2623.518176] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2623.721233] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2628.806798] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2628.960324] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2633.190675] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2633.193365] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2633.211780] Lustre: Skipped 1 previous similar message [ 2633.234228] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2633.239862] Lustre: Skipped 1 previous similar message [ 2637.233096] Lustre: server umount lustre-OST0000 complete [ 2647.472954] Lustre: server umount lustre-MDT0000 complete [ 2648.548836] LustreError: 90984:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2648.562963] LustreError: 90984:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 2651.459203] LustreError: 90964:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790736996 with bad export cookie 5767902620884601986 [ 2651.462036] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2651.470985] LustreError: 90964:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 2652.098333] Lustre: server umount lustre-MDT0001 complete [ 2661.208607] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 22:56:44 (1790737004) [ 2668.186574] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2674.183375] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 2683.257737] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2683.365329] LustreError: 95354:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2683.394408] LustreError: 95354:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 2688.833970] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 2696.882033] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2702.948409] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2710.453868] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2710.691818] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2710.701210] Lustre: Skipped 4 previous similar messages [ 2717.770096] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 2726.743752] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2732.842726] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2733.170153] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2739.759611] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2740.054191] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2748.058366] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2748.349642] Lustre: lustre-OST0001: new disk, initializing [ 2748.352215] Lustre: Skipped 2 previous similar messages [ 2748.354696] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2748.358088] Lustre: Skipped 3 previous similar messages [ 2748.364748] Lustre: 98723:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2754.710329] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 2758.238183] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2758.249953] Lustre: Skipped 2 previous similar messages [ 2758.257451] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2758.385527] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2763.923467] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2768.425355] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2768.670967] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2773.276668] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2773.488925] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2791.953721] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2813.250450] Lustre: Failing over lustre-MDT0000 [ 2813.827521] Lustre: server umount lustre-MDT0000 complete [ 2814.964312] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2814.975695] LustreError: 96342:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2814.977706] Lustre: Skipped 5 previous similar messages [ 2815.011645] LustreError: 96342:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2830.305539] Lustre: 66749:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790737159/real 1790737159] req@000000001177183c x1877723015564288/t0(0) o400->MGC192.168.201.145@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1790737175 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2830.377461] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2838.958557] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2838.961313] LDISKFS-fs (dm-0): recovery complete [ 2838.967851] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2841.069780] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2842.149920] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2842.155705] Lustre: 95348:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2846.043279] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 2846.180283] Lustre: 99325:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2846.203638] Lustre: 99325:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 1 previous similar message [ 2846.224180] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2846.411161] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2846.730278] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2847.077419] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2847.094458] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 2847.409906] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2847.423087] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 2847.442680] Lustre: 101574:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2847.465443] Lustre: 101574:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 7 previous similar messages [ 2848.116596] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2848.126501] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 2 previous similar messages [ 2848.460665] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2848.472739] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 2 previous similar messages [ 2849.738674] Lustre: 101574:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2849.763278] Lustre: 101574:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 6 previous similar messages [ 2850.406582] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2850.419170] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 6 previous similar messages [ 2850.744361] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2850.760160] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 6 previous similar messages [ 2853.953449] Lustre: 101574:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2853.984450] Lustre: 101574:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 12 previous similar messages [ 2854.644321] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2854.651166] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 2854.976163] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2854.984459] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 2861.995985] Lustre: 101574:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2862.016647] Lustre: 101574:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 24 previous similar messages [ 2862.664444] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2862.683088] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 24 previous similar messages [ 2863.000464] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2863.013748] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 24 previous similar messages [ 2878.241101] Lustre: 101574:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2878.260226] Lustre: 101574:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 50 previous similar messages [ 2878.906189] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2878.934723] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 50 previous similar messages [ 2879.264955] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2879.279953] LustreError: 101574:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 50 previous similar messages [ 2902.217665] Lustre: 101574:0:(ldlm_lib.c:2428:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2902.252989] Lustre: 101574:0:(ldlm_lib.c:2438:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2902.289456] Lustre: 101574:0:(genops.c:1600:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 4ba5498e-fb46-47ca-9c1c-9cff14728d8f@192.168.201.45@tcp [ 2902.328789] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2902.344810] Lustre: 101574:0:(ldlm_lib.c:1948:abort_req_replay_queue()) @@@ aborted: req@000000005b492da3 x1877722994378624/t0(8589934771) o101->4ba5498e-fb46-47ca-9c1c-9cff14728d8f@192.168.201.45@tcp:322/0 lens 592/0 e 0 to 0 dl 1790737257 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2902.412597] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2902.466544] Lustre: lustre-MDT0000: Denying connection for new client 4ba5498e-fb46-47ca-9c1c-9cff14728d8f (at 192.168.201.45@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2902.531955] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2902.571967] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2902.574249] Lustre: Skipped 2 previous similar messages [ 2902.583168] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2902.683133] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 2902.684283] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2922.995567] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2923.006517] Lustre: Skipped 3 previous similar messages [ 2926.229662] Lustre: server umount lustre-OST0000 complete [ 2932.708103] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2932.711609] LustreError: Skipped 1 previous similar message [ 2937.374768] Lustre: server umount lustre-OST0001 complete [ 2943.459137] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2943.490298] Lustre: Skipped 6 previous similar messages [ 2947.728801] Lustre: server umount lustre-MDT0000 complete [ 2948.583954] LustreError: 95353:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2948.606594] LustreError: 95353:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 37 previous similar messages [ 2951.561912] LustreError: 95335:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790737296 with bad export cookie 5767902620884653198 [ 2951.570381] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2951.581125] LustreError: 95335:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2952.019365] Lustre: server umount lustre-MDT0001 complete [ 2960.628564] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 23:01:43 (1790737303) [ 2967.549448] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2967.555894] Lustre: *** cfs_fail_loc=197, val=0*** [ 2983.142268] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2988.320616] LustreError: 103565:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.145@tcp: failed processing log, type 4: rc = -110 [ 2993.495757] Lustre: *** cfs_fail_loc=197, val=0*** [ 2993.499761] Lustre: Skipped 65 previous similar messages [ 3014.049935] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3014.063268] Lustre: Skipped 2 previous similar messages [ 3019.347975] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 3026.392695] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3028.380563] Lustre: server umount lustre-OST0000 complete [ 3035.459741] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 23:02:58 (1790737378) [ 3055.300159] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3060.191370] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 3068.865977] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3072.492678] LustreError: 105214:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3072.505103] LustreError: 105214:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3072.527593] LustreError: 105214:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 17, retries 0, failed: rc = -5 [ 3073.000819] Lustre: server umount lustre-MDT0000 complete [ 3105.481118] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_hostid [ 3115.163075] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 3150.737926] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3151.034476] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3151.052890] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3151.106385] Lustre: lustre-MDT0000: new disk, initializing [ 3151.170565] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3155.275575] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 3166.000088] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3166.169499] Lustre: 109389:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3166.468863] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3172.318697] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 3181.392571] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3187.237910] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3194.496454] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3194.713700] Lustre: 111357:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3200.936231] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 3204.355580] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3209.720257] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3213.938122] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3214.225178] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3219.588317] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3219.959397] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3225.057753] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3225.064517] LustreError: Skipped 1 previous similar message [ 3225.072022] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3225.072648] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3225.100120] Lustre: Skipped 6 previous similar messages [ 3228.632752] Lustre: server umount lustre-OST0000 complete [ 3238.786560] Lustre: server umount lustre-MDT0000 complete [ 3240.418420] LustreError: 109402:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3240.435989] LustreError: 109402:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 3242.017550] LustreError: 109383:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790737586 with bad export cookie 5767902620884663796 [ 3242.031355] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3242.389821] Lustre: server umount lustre-MDT0001 complete [ 3251.421874] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 23:06:34 (1790737594) [ 3260.108643] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [ 3262.840052] Key type lgssc unregistered [ 3263.118158] LNet: 113843:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3263.138213] LNetError: 113843:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3263.159086] LNet: Removed LNI 192.168.201.145@tcp [ 3264.104700] Key type .llcrypt unregistered [ 3264.106413] Key type ._llcrypt unregistered [ 3282.503402] Key type ._llcrypt registered [ 3282.506260] Key type .llcrypt registered [ 3282.595293] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_hostid [ 3297.466019] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 3298.735376] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3298.758490] alg: No test for adler32 (adler32-zlib) [ 3299.783824] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [ 3300.033331] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [ 3301.704618] Key type lgssc registered [ 3302.794862] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3345.782432] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 3345.839443] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3347.342198] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3347.387911] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3347.522469] Lustre: lustre-MDT0000: new disk, initializing [ 3347.602265] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3347.626740] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3354.028697] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 3364.544090] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3364.844512] Lustre: lustre-OST0000: new disk, initializing [ 3364.847430] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3364.852137] Lustre: Skipped 1 previous similar message [ 3364.858875] Lustre: 118033:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3364.932047] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3370.759627] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 3374.743703] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3374.751942] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3374.782869] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3380.127618] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3380.261200] Lustre: lustre-OST0001: new disk, initializing [ 3380.264530] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3380.272311] Lustre: 118897:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3380.329777] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3380.752545] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3380.782760] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3380.819624] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3386.286265] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 3396.077950] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3396.095304] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3396.100389] Lustre: Skipped 1 previous similar message [ 3401.186390] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3401.190770] Lustre: Skipped 1 previous similar message [ 3401.391600] Lustre: server umount lustre-MDT0000 complete [ 3408.198281] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3412.765032] LustreError: 117219:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790737757 with bad export cookie 14977436421521636175 [ 3412.774036] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3412.775104] LustreError: 117219:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3412.844297] Lustre: server umount lustre-OST0000 complete [ 3416.739986] Lustre: server umount lustre-OST0001 complete [ 3425.515343] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 23:09:28 (1790737768) [ 3445.920430] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 23:09:48 (1790737788) [ 3456.203972] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 3481.537602] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 23:10:24 (1790737824) [ 3522.697348] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_hostid [ 3533.250752] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 3569.552379] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3569.795968] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3569.827973] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3569.895546] Lustre: lustre-MDT0000: new disk, initializing [ 3569.981260] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3569.997410] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3574.756707] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 3586.258827] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3586.435346] Lustre: 126649:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3586.524276] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3586.537960] Lustre: Skipped 1 previous similar message [ 3586.649436] Lustre: lustre-MDT0001: new disk, initializing [ 3586.783441] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3586.833519] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3586.847910] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3591.907497] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 3600.665930] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3606.319485] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3612.911308] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3613.092069] Lustre: lustre-OST0000: new disk, initializing [ 3613.094669] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3613.099231] Lustre: 128615:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3613.174557] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3619.681414] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 3620.470049] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3620.486291] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3620.641788] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3628.992731] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3634.269437] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3634.641508] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3639.960855] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3640.212486] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3646.441841] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3646.442627] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3646.447614] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3646.454178] Lustre: Skipped 2 previous similar messages [ 3646.463611] LustreError: Skipped 1 previous similar message [ 3648.328233] Lustre: server umount lustre-OST0000 complete [ 3651.558716] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3651.569033] Lustre: Skipped 1 previous similar message [ 3651.572157] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3651.579560] Lustre: Skipped 2 previous similar messages [ 3656.674748] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3656.688751] Lustre: Skipped 1 previous similar message [ 3657.830127] Lustre: server umount lustre-MDT0000 complete [ 3661.748810] LustreError: 126641:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790738006 with bad export cookie 14977436421521637400 [ 3661.756614] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3661.763983] LustreError: 126641:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3662.034274] Lustre: server umount lustre-MDT0001 complete [ 3673.000211] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 23:13:35 (1790738015) [ 3683.002870] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3683.814967] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3688.936092] LustreError: 131028:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3688.956137] LustreError: 131028:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 3689.624616] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 3694.062986] LustreError: 131029:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3698.167671] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3702.852342] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 3710.569984] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3717.065485] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3723.491575] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3723.791992] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3723.801882] Lustre: Skipped 1 previous similar message [ 3731.695622] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 3739.337470] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3744.789288] Lustre: *** cfs_fail_loc=159, val=0*** [ 3761.193899] Lustre: lustre-MDT0000: Client e0ceae48-a5da-40ab-b124-cb4e742c5765 (at 192.168.201.45@tcp) reconnecting [ 3762.999150] Lustre: *** cfs_fail_loc=159, val=0*** [ 3763.003092] Lustre: Skipped 5 previous similar messages [ 3778.666665] Lustre: lustre-MDT0000: Client e0ceae48-a5da-40ab-b124-cb4e742c5765 (at 192.168.201.45@tcp) reconnecting [ 3785.186961] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3785.201258] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3785.217465] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3786.213232] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3789.093415] Lustre: server umount lustre-OST0000 complete [ 3795.937195] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3795.940184] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3795.945325] Lustre: Skipped 1 previous similar message [ 3798.508958] Lustre: server umount lustre-MDT0000 complete [ 3801.571041] LustreError: 131029:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3801.581781] LustreError: 131029:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 3801.668654] LustreError: 131009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790738146 with bad export cookie 14977436421521638282 [ 3801.670079] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3801.677890] LustreError: 131009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3802.043632] Lustre: server umount lustre-MDT0001 complete [ 3812.024646] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [ 3814.659935] Key type lgssc unregistered [ 3814.972842] LNet: 134801:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3814.982907] LNetError: 134801:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3814.997963] LNet: Removed LNI 192.168.201.145@tcp [ 3815.995371] Key type .llcrypt unregistered [ 3815.997102] Key type ._llcrypt unregistered [ 3839.776642] Key type ._llcrypt registered [ 3839.778778] Key type .llcrypt registered [ 3839.975322] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 3841.136439] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3841.265112] alg: No test for adler32 (adler32-zlib) [ 3842.269753] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [ 3842.428099] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [ 3844.080968] Key type lgssc registered [ 3845.610356] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3855.272394] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 3855.340564] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3856.870520] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3860.705572] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 3867.109797] LustreError: 136238:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3867.131034] LustreError: 136238:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 3870.454183] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3870.828282] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3876.188822] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 3884.766168] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3891.463574] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3897.292804] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3897.535396] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3902.671138] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3904.997099] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 3911.539340] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3924.361055] Lustre: *** cfs_fail_loc=159, val=0*** [ 3939.926511] Lustre: lustre-MDT0000: Client e77ce673-1fc9-4854-b50c-f9b44fff7332 (at 192.168.201.45@tcp) reconnecting [ 3942.101226] Lustre: *** cfs_fail_loc=159, val=0*** [ 3958.372461] Lustre: lustre-MDT0001: Client e77ce673-1fc9-4854-b50c-f9b44fff7332 (at 192.168.201.45@tcp) reconnecting [ 3959.748966] Lustre: *** cfs_fail_loc=159, val=0*** [ 3959.762869] Lustre: Skipped 4 previous similar messages [ 3976.274463] Lustre: lustre-MDT0001: Client e77ce673-1fc9-4854-b50c-f9b44fff7332 (at 192.168.201.45@tcp) reconnecting [ 3980.712653] Lustre: 139278:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3983.394325] Lustre: *** cfs_fail_loc=159, val=0*** [ 3983.402035] Lustre: Skipped 2 previous similar messages [ 3998.849920] Lustre: lustre-MDT0000: Client ec8b7788-434c-430b-aadb-2d27a4be8a61 (at 192.168.201.45@tcp) reconnecting [ 4001.210957] Lustre: *** cfs_fail_loc=159, val=0*** [ 4001.225824] Lustre: Skipped 15 previous similar messages [ 4016.802643] Lustre: lustre-MDT0000: Client ec8b7788-434c-430b-aadb-2d27a4be8a61 (at 192.168.201.45@tcp) reconnecting [ 4018.516649] Lustre: 139569:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4023.791458] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4023.797766] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4023.799266] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4023.816902] Lustre: Skipped 1 previous similar message [ 4028.899497] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4028.907981] Lustre: Skipped 2 previous similar messages [ 4029.871642] Lustre: server umount lustre-OST0000 complete [ 4034.019370] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4034.041029] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4034.059140] Lustre: Skipped 1 previous similar message [ 4039.137793] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4039.144691] Lustre: Skipped 1 previous similar message [ 4039.354504] Lustre: server umount lustre-MDT0000 complete [ 4043.277462] LustreError: 136217:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790738388 with bad export cookie 18336408189629916670 [ 4043.280059] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4043.287253] LustreError: 136217:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4043.687711] Lustre: server umount lustre-MDT0001 complete [ 4054.677710] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [ 4057.115873] Key type lgssc unregistered [ 4057.437734] LNet: 140620:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4057.448689] LNetError: 140620:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4057.478641] LNet: Removed LNI 192.168.201.145@tcp [ 4058.318918] Key type .llcrypt unregistered [ 4058.320956] Key type ._llcrypt unregistered [ 4081.405350] Key type ._llcrypt registered [ 4081.407906] Key type .llcrypt registered [ 4081.510094] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 4082.662893] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4082.715683] alg: No test for adler32 (adler32-zlib) [ 4083.746642] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [ 4083.957351] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [ 4085.594570] Key type lgssc registered [ 4086.757147] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4093.309436] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4093.322797] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4094.743503] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4099.023099] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 4104.167866] LustreError: 142050:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4104.201426] LustreError: 142050:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4108.668602] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4109.148827] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4114.805224] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 4122.714979] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4128.959433] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4134.889634] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4135.216792] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4139.315446] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4140.421814] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 4147.113269] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4153.656967] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4156.653314] Lustre: 144635:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4158.722739] Lustre: 144683:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4159.878927] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4160.632447] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4163.533767] Lustre: 144829:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4164.336526] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4165.189615] Lustre: 144927:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4169.185561] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4169.191168] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4169.202265] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4169.226368] Lustre: Skipped 1 previous similar message [ 4169.238112] Lustre: Skipped 1 previous similar message [ 4174.311656] LustreError: 144170:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4174.328220] LustreError: 144170:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4174.536710] Lustre: server umount lustre-OST0000 complete [ 4179.429796] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4179.432400] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4179.452835] Lustre: Skipped 1 previous similar message [ 4179.473160] Lustre: Skipped 1 previous similar message [ 4184.438463] Lustre: server umount lustre-MDT0000 complete [ 4184.545311] LustreError: 142045:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4184.565239] LustreError: 142045:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4187.835186] LustreError: 142031:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790738532 with bad export cookie 13576145370920602994 [ 4187.851311] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4187.854665] LustreError: 142031:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4188.086169] Lustre: server umount lustre-MDT0001 complete [ 4197.745562] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [ 4200.756726] Key type lgssc unregistered [ 4201.003433] LNet: 145986:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4201.009633] LNetError: 145986:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4201.036915] LNet: Removed LNI 192.168.201.145@tcp [ 4201.763513] Key type .llcrypt unregistered [ 4201.765859] Key type ._llcrypt unregistered [ 4223.667026] Key type ._llcrypt registered [ 4223.669473] Key type .llcrypt registered [ 4223.790474] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 4224.982605] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4225.081384] alg: No test for adler32 (adler32-zlib) [ 4226.171309] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [ 4226.507833] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [ 4228.187885] Key type lgssc registered [ 4229.481881] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4236.985402] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4237.009610] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4238.466925] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4243.499754] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 4249.072513] LustreError: 147414:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4249.119547] LustreError: 147414:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4253.020595] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4253.695831] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4258.968194] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 4265.682203] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4271.418626] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4277.781455] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4278.043278] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4279.106177] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4283.186165] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4284.090829] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 4290.900836] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4297.893731] Lustre: *** cfs_fail_loc=159, val=0*** [ 4314.198275] Lustre: lustre-MDT0000: Client 7c8008bb-b2d3-4f05-9fb2-f780771e78ed (at 192.168.201.45@tcp) reconnecting [ 4319.207211] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4319.223101] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4319.225941] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4319.241631] Lustre: Skipped 1 previous similar message [ 4321.319358] Lustre: server umount lustre-OST0000 complete [ 4325.348093] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4325.362170] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4325.369017] Lustre: Skipped 1 previous similar message [ 4325.373171] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4329.444263] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4329.448279] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4329.470361] Lustre: Skipped 1 previous similar message [ 4330.923850] Lustre: server umount lustre-MDT0000 complete [ 4334.570882] LustreError: 147413:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4334.592081] LustreError: 147413:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4334.694734] LustreError: 147395:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790738679 with bad export cookie 990903225119175545 [ 4334.703342] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4334.704112] LustreError: 147395:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4334.944880] Lustre: server umount lustre-MDT0001 complete [ 4346.039596] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [ 4349.002522] Key type lgssc unregistered [ 4349.405909] LNet: 151080:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4349.411955] LNetError: 151080:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4349.426149] LNet: Removed LNI 192.168.201.145@tcp [ 4350.554169] Key type .llcrypt unregistered [ 4350.556487] Key type ._llcrypt unregistered [ 4378.642208] Key type ._llcrypt registered [ 4378.652342] Key type .llcrypt registered [ 4378.826492] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 4380.327392] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4380.517112] alg: No test for adler32 (adler32-zlib) [ 4381.605856] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [ 4381.871698] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [ 4383.568239] Key type lgssc registered [ 4384.729862] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4392.880859] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4392.895961] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4394.258029] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4398.913857] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 4404.709078] LustreError: 152511:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4404.730023] LustreError: 152511:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4408.120281] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4408.519929] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4413.295131] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 4422.668330] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4429.206749] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4437.080718] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4437.631102] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4439.698101] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4445.730269] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 4453.623598] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4454.012702] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4473.898903] Lustre: lustre-OST0000: evicting 28c5ef61-ca12-4c3b-bce9-4b4fbafeaf2d (at 192.168.201.45@tcp) by administrative request [ 4473.902709] Lustre: Skipped 1 previous similar message [ 4505.582691] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4505.584218] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4505.599086] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4505.636729] Lustre: Skipped 1 previous similar message [ 4510.764414] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4510.774570] Lustre: Skipped 2 previous similar messages [ 4515.810963] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4515.820749] Lustre: Skipped 1 previous similar message [ 4519.393829] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4519.706980] Lustre: server umount lustre-OST0000 complete [ 4526.055559] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4526.060280] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4526.067899] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4526.078889] Lustre: Skipped 1 previous similar message [ 4526.097390] Lustre: Skipped 1 previous similar message [ 4530.577711] Lustre: server umount lustre-MDT0000 complete [ 4531.182242] LustreError: 153242:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4531.205371] LustreError: 153242:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4534.625737] LustreError: 154364:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790738879 with bad export cookie 10858624680034231584 [ 4534.630071] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4534.643331] LustreError: 154364:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4535.111153] Lustre: server umount lustre-MDT0001 complete [ 4550.729189] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [ 4553.584343] Key type lgssc unregistered [ 4553.939828] LNet: 156593:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4553.944939] LNetError: 156593:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4553.956833] LNet: Removed LNI 192.168.201.145@tcp [ 4554.925898] Key type .llcrypt unregistered [ 4554.929390] Key type ._llcrypt unregistered [ 4638.240488] Key type ._llcrypt registered [ 4638.242844] Key type .llcrypt registered [ 4638.348842] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 4639.285614] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4639.397070] alg: No test for adler32 (adler32-zlib) [ 4640.377675] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [ 4640.593217] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [ 4642.280229] Key type lgssc registered [ 4643.838752] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4652.017863] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4652.035422] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4653.651704] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4659.071580] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 4663.778441] LustreError: 159095:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4663.802599] LustreError: 159095:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4668.232197] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4668.928565] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4672.601608] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 4681.482773] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4687.194528] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4693.459470] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4693.847624] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4699.010592] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4700.147300] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 4707.117173] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4713.334252] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4723.684661] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4723.687060] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4723.752040] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4723.763674] Lustre: Skipped 1 previous similar message [ 4728.447494] Lustre: server umount lustre-OST0000 complete [ 4733.923556] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4733.927571] Lustre: Skipped 1 previous similar message [ 4733.930375] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4733.934085] Lustre: Skipped 1 previous similar message [ 4739.041695] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4739.051957] Lustre: Skipped 1 previous similar message [ 4743.139619] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4743.142862] Lustre: Skipped 1 previous similar message [ 4746.210759] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4746.598085] Lustre: server umount lustre-MDT0000 complete [ 4748.271091] LustreError: 159096:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4748.309552] LustreError: 159096:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4750.680164] LustreError: 159078:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790739095 with bad export cookie 17597421316071796481 [ 4750.688909] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4750.689851] LustreError: 159078:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4751.095248] Lustre: server umount lustre-MDT0001 complete [ 4764.271892] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [ 4767.865463] Key type lgssc unregistered [ 4768.284780] LNet: 162658:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4768.301124] LNetError: 162658:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4768.340296] LNet: Removed LNI 192.168.201.145@tcp [ 4769.295691] Key type .llcrypt unregistered [ 4769.299069] Key type ._llcrypt unregistered [ 4793.684122] Key type ._llcrypt registered [ 4793.692176] Key type .llcrypt registered [ 4793.809282] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 4795.118077] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4795.157094] alg: No test for adler32 (adler32-zlib) [ 4796.273832] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [ 4796.540567] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [ 4798.200210] Key type lgssc registered [ 4799.379249] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4813.366780] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 23:32:36 (1790739156) [ 4829.950355] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_hostid [ 4839.029801] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 4874.669572] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4874.697065] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4876.128748] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4876.165203] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4876.299782] Lustre: lustre-MDT0000: new disk, initializing [ 4876.437788] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4876.490366] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4881.850296] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 4893.485886] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4893.678296] Lustre: 166940:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4893.767655] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4893.773709] Lustre: Skipped 1 previous similar message [ 4893.894439] Lustre: lustre-MDT0001: new disk, initializing [ 4894.134848] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4894.222907] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4894.241257] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4899.199294] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 4909.951346] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4916.227649] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4923.342849] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4923.645747] Lustre: lustre-OST0000: new disk, initializing [ 4923.652745] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4923.687522] Lustre: 168909:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4923.813301] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4929.663218] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 4931.149205] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4931.170188] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4931.312799] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4938.779638] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4946.914850] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4946.916672] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4946.923072] LustreError: Skipped 1 previous similar message [ 4946.923404] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4946.938533] Lustre: Skipped 1 previous similar message [ 4952.035097] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4952.043375] Lustre: Skipped 2 previous similar messages [ 4957.159695] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4957.162693] Lustre: Skipped 1 previous similar message [ 4961.248189] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4961.408905] Lustre: server umount lustre-OST0000 complete [ 4965.860996] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4965.868774] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4965.882878] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4967.394866] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4972.514042] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4972.518390] Lustre: Skipped 3 previous similar messages [ 4979.168167] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4979.341042] Lustre: server umount lustre-MDT0000 complete [ 4983.246184] LustreError: 167735:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790739328 with bad export cookie 10161554992595222256 [ 4983.269377] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4983.712750] Lustre: server umount lustre-MDT0001 complete [ 4994.987526] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [ 4998.098083] Key type lgssc unregistered [ 4998.432712] LNet: 170814:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4998.442368] LNetError: 170814:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4998.468863] LNet: Removed LNI 192.168.201.145@tcp [ 4999.280206] Key type .llcrypt unregistered [ 4999.284395] Key type ._llcrypt unregistered [ 5023.882265] Key type ._llcrypt registered [ 5023.885078] Key type .llcrypt registered [ 5023.984836] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 5025.191269] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5025.328043] alg: No test for adler32 (adler32-zlib) [ 5026.454812] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [ 5026.866126] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [ 5028.584317] Key type lgssc registered [ 5029.775867] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5039.198552] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 5039.217535] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5040.717696] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5044.827441] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 5050.850196] LustreError: 172250:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5050.877516] LustreError: 172250:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 5053.448398] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5053.866703] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5059.350142] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 5068.192372] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5074.980649] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5083.280119] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5083.638617] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5085.706060] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5090.141870] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 5098.839788] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5106.043329] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5106.061662] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5106.075266] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5106.090574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.117332] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5107.239207] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5107.261071] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5107.286600] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5107.293064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.329557] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5108.247115] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5108.259953] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5108.272103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.036873] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5109.042378] Lustre: Skipped 1 previous similar message [ 5109.046183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.057832] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5109.065886] Lustre: Skipped 1 previous similar message [ 5109.847347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.838680] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5110.844281] Lustre: Skipped 2 previous similar messages [ 5110.862652] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5110.873817] Lustre: Skipped 2 previous similar messages [ 5110.878178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.868188] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5111.871050] Lustre: Skipped 2 previous similar messages [ 5111.879602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.887328] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5111.891991] Lustre: Skipped 2 previous similar messages [ 5113.184105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.140435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.108426] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5115.113940] Lustre: Skipped 3 previous similar messages [ 5115.125552] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5115.132458] Lustre: Skipped 3 previous similar messages [ 5115.135875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.978581] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5115.985404] Lustre: Skipped 3 previous similar messages [ 5115.988460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.999994] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5116.004279] Lustre: Skipped 3 previous similar messages [ 5116.932262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.929641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.028198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.023350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.995467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.670258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.766422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.512673] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5123.519711] Lustre: Skipped 8 previous similar messages [ 5123.545993] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5123.559619] Lustre: Skipped 8 previous similar messages [ 5123.572272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.484472] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5124.487142] Lustre: Skipped 8 previous similar messages [ 5124.490096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.494932] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5124.500326] Lustre: Skipped 8 previous similar messages [ 5125.406050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.312548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.097823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.045981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.738958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.366489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.127573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.957947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.618543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.532630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.252177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.912806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.951397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.769686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.983426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.171962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.214277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.423279] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5140.428602] Lustre: Skipped 18 previous similar messages [ 5140.443450] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5140.457964] Lustre: Skipped 18 previous similar messages [ 5140.472542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.522657] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5140.539795] Lustre: Skipped 17 previous similar messages [ 5141.628780] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5141.631599] Lustre: Skipped 18 previous similar messages [ 5141.635147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.704390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.036106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.983848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.944441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.720937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.105256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.231927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.619528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.859658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.790432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.572752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.312517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.175778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.169287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.809795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.713326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.889653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.802906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.919386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.094786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.875290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.850585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.864939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.852699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.530639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.188380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.005161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.814290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.577640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.410073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.268157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.128877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.777298] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5172.779960] Lustre: Skipped 33 previous similar messages [ 5172.790701] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5172.796794] Lustre: Skipped 33 previous similar messages [ 5172.799764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.809613] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5172.824509] Lustre: Skipped 33 previous similar messages [ 5173.383402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.023700] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5174.028628] Lustre: Skipped 34 previous similar messages [ 5174.031780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.824665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.448986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.364839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.142600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.930159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.633168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.443493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.321565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.091866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.852986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.570017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.263640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.132754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.996824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.837667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.606944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.434406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.315291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.334535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.233092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.240647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.192802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.117832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.010950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.961299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.956971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.904781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.750969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.513516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.427098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.299320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.154420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.108570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.844988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.697060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.782950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.822918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.871697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.626579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.449969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.204956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.981172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.774831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.718262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.481869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.450935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.504815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.434135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.327639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.199509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.273670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.327116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.307663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.220753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.231070] LustreError: 172269:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5222.242669] LustreError: 172269:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5222.254590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.132510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.983656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.764226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.745705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.567092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.554460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.298972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.998373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.880278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.952498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.833368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.867512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.737601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.591856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.390601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.166681] LustreError: 172269:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5236.181391] LustreError: 172269:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5236.192743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.987738] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5236.991909] Lustre: Skipped 73 previous similar messages [ 5237.011536] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5237.018901] Lustre: Skipped 73 previous similar messages [ 5237.025763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.035095] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5237.040788] Lustre: Skipped 73 previous similar messages [ 5238.016289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.970551] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5238.977514] Lustre: Skipped 73 previous similar messages [ 5238.982846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.117664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.958529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.112078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.968584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.113137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.277866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.302315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.183377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.933250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.637445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.372909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.258856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.080051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.971947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.716801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.679493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.655600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.587507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.769425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.714647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.929459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.049962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.049225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.149095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.724161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.562828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.511503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.579839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.287036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.133291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.025797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.005983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.936204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.933322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.650178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.487934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.462788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.456637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.360827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.598045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.534872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.499196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.577416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.577787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.808944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.691392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.766391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.637220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.461445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.096426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.908235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.419391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.164987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.763800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.592592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.530387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.628641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.488804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.372215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.159267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.007772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.946788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.910284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.746115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.613153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.474430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.218742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.827412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.529794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.347703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.223388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.212170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.212423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.521421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.801716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.706577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.679159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.645286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.750861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.675824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.810121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.946549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.054930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.920149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.306198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.631206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.423546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.605043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.817443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.110333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.188227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.936447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.972911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.013080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.233803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.096217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.044078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.228292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.977539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.851501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.578350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.448469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.526714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.523764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.676634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.533134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.384679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.400691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.236079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.155438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.025970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.853314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.949659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.907302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.924516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.881188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.764474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.631329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.661397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.527854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.775277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.739692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.572310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.496837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.523926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.284832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.173981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.403208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.360251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.104382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.813393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.695346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.603131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.538337] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5365.546327] Lustre: Skipped 135 previous similar messages [ 5365.560325] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5365.571811] Lustre: Skipped 135 previous similar messages [ 5365.575484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.582518] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5365.585056] Lustre: Skipped 135 previous similar messages [ 5366.391229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.429909] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5367.441123] Lustre: Skipped 135 previous similar messages [ 5367.444802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.163190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.031647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.754325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.583812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.427798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.355323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.043287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.792268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.441672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.293834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.070619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.916296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.651508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.607106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.473477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.271943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.046655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.196976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.211316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.100339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.925713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.996656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.971623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.230096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.307613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.407078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.268149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.114413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.991538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.807766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.965910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.146059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.001900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.911652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.878474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.976077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.918225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.938987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.946221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.978415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.824780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.443661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.095580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.059146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.125476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.216025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.119571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.223599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.179709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.310514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.198974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.154787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.049340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.912464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.679877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.533740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.641539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.597718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.725803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.476965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.574166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.415960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.347874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.153214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.875581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.853394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.888816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.748131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.637582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.438804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.255690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.409641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.311070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.251801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.112683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.041580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.964161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.292789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.438579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.481807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.603092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.903351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.215222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.280954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.358139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.284824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.263794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.131391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.989655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.695620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.628774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.579441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.230428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.096387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.044852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.853304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.741952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.503986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.496897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.281814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.236260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.025993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.762171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.500258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.767701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.837519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.231404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.270563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.139038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.098550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.076596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.103123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.799516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.833300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.088170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.016045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.057942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.970459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.833335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.526643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.319874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.436532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.162737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.167550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.311769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.276908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.368096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.275444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.169305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.975065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.820477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.798535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.539599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.318802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.270358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.196761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.257954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.433792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.397308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.296915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.315852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.424497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.218463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.129438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.063847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.088278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.938248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.949118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.881710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.831105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.688958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.488061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.372934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.488479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.299333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.236230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.202974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.079663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.821814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.789954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.759078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.738974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.589297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.657884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.567705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.482512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.399146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.526712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.635270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.693941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.900965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.965691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.865383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.919356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.072304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.113833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.277052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.650593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.984592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.215201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.127445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.996877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.354748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.649092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.731608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.822917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.975418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.833905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.115494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.773583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.056466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.409904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.677887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.769869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.150667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.578682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.589261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.856133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.890606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.846943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.776384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.836562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.764699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.044904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.185398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.470991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.538620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.700996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.745697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.920175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.811286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.003868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.930106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.793091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.684856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.620491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.589424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.646788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.612218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.720932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.723214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.785684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.818560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.066440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.117977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.187709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.236365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.089990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.136297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.128339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.034805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.795546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.694225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.530751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.345180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.164530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.222419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.034255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.008434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.974435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.132521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.131814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.433469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.699875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.849415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.987948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.410234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.458890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.258431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.133618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.843714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.758373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.776117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.866467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.872761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.889699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.896388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.940019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.969608] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5621.982441] Lustre: Skipped 260 previous similar messages [ 5621.992682] LustreError: 172269:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5622.009146] LustreError: 172269:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5622.043811] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5622.061878] Lustre: Skipped 260 previous similar messages [ 5622.066056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.080716] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5622.087313] Lustre: Skipped 260 previous similar messages [ 5623.025717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.884814] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5623.888798] Lustre: Skipped 260 previous similar messages [ 5623.892850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.023649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.194996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.081187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.065296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.882056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.664582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.586619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.312444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.233559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.146469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.110526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.103287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.050346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.934441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.942746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.783293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.702395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.402025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.263150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.889167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.734299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.427719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.357973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.323401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.145063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.011311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.993639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.005277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.831230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.642674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.608979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.739616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.611231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.809937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.660193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.574226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.664820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.432373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.657799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.776949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.694234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.550489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.473992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.312289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.409277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.270493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.438938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.640875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.115779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.297297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.432813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.978932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.360027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.552902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.316410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.629597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.999308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.324379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.251577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.399608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.363865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.466832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.886372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.155996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.460429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.472287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.650798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.785162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.885890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.855554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.739199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.803881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.970341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.096061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.056641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.368564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.787217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.078267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.389847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.316133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.583702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.622100] LustreError: 172269:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5709.629108] LustreError: 172269:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5709.649835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.008349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.212165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.165694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.028928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.837959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.687467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.476798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.278399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.080260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.988850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.930623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.993645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.816534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.809221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.693852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.691910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.551949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.431806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.151519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.177633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.094460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.094492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.140323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.209264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.164904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.157607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.050608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.934465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.993748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.786081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.557822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.384335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.155038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.013068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.918637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.748681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.747325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.604521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.333255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.249954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.078063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.063526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.990145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.874934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.681638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.743930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.536150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.499634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.294073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.212770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.021403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.928778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.791419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.716245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.567701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.456162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.221548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.020756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.977723] LustreError: 172269:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5762.994028] LustreError: 172269:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5763.007855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.115271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.447305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.741793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.780825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.825713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.770725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.873934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.865173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.599955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.388822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.366720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.253047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.128167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.084235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.060226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.747910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.714211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.509667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.440108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.233132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.064919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.693219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.407356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.192088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.128674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.039834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.981132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.988865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.630554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.442593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.243378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.142502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.933109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.881104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.795475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.564393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.299327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.348868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.215393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.000226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.855733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.868120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.788604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.743996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.751876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.686210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.624469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.446967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.313217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.057778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.750321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.570473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.623417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.480548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.435074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.400548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.197754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.252910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.108818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.418448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.376107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.659893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.815822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.646795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.586284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.330215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.207109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.999048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.995784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.800717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.707844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.495707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.498805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.316147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.261387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.039804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.953708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.777023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.649691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.338782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.206065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.195447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.106952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.000475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.909040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.896407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.848193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.710691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.565909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.403520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.381483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.238361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.214670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.952861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.860757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.745648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.675845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.620801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.427360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.387989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.519223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.611453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.985428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.864498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.189717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.356860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.215091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.230390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.239917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.410278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.593793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.501935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.536926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.428327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.543854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.527570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.671395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.572625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.895149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.998462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.153468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.300415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.658958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.821526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.155771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.552557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.716647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.575492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.496806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.650896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.573652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.530146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.786578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.965033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.027327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.910847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.559588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.470364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.263853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.349386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.415362] LustreError: 172269:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5897.429044] LustreError: 172269:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5897.447159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.586743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.751267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.656360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.726348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.575109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.436565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.347766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.241364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.011880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.880890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.706059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.614766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.397356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.519492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.450872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.357974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.185213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.143053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.853400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.590879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.407936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.325841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.014858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.774375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.641199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.477236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.225795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.048699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.693265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.451678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.294256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.146636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.094263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.852812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.723793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.492519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.545917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.400475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.698211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.556171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.500435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.391303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.283243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.156889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.109093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.877576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.851903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.740686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.048717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.994383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.305320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.449485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.477744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.754148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.751685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.829438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.700643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.950129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.892064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.244158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.142365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.269148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.391429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.817829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.951613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.095036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.442164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.295301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.300414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.251294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.253795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.126175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.038790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.283731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.521800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.439740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.365182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.355115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.352686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.238874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.309706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.289930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.347381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.285095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.278368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.152382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.074236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.972393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.851617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.707739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.758515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.622645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.456378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.335844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.248194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.099309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.090820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.065030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.094498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.935334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.886359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.042685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.125617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.125493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.111276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.918211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.623654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.281540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.268247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.042929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.974248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.049636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.853046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.677295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.638755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.512352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.527572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.498400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.461201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.355246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.293353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.203423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.487993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.674942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.761872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.949699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.932942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.734822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.788299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.789264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.708447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.612033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.665315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.779874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.873228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.114878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.328471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.392486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.441527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.674277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.718612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.534211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.530848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.955934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.014911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.155543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.305499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.545836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.982732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.990688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.983622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.892796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.025557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.159802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.220041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.187642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.087926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.144874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.416883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.335570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.317949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.372321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.628952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.760729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.806102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.180421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.592081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.924498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.979608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.583526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.072545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.707292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.086034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.949699] LustreError: 177026:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6387.955941] LustreError: 172269:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6809.070298] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6809.075966] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6809.083969] Lustre: Skipped 457 previous similar messages [ 6809.099079] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6811.110231] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6811.126714] Lustre: Skipped 1 previous similar message [ 6814.199399] Lustre: server umount lustre-OST0000 complete [ 6820.321234] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6820.330328] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6825.416344] Lustre: server umount lustre-MDT0000 complete [ 6826.475083] LustreError: 172249:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6826.496840] LustreError: 172249:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 6829.811269] LustreError: 172229:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790741174 with bad export cookie 6032993784594694110 [ 6829.820303] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6829.843521] LustreError: 172229:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6830.165940] Lustre: server umount lustre-MDT0001 complete [ 6841.356408] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [ 6843.937434] Key type lgssc unregistered [ 6844.193706] LNet: 227490:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6844.201713] LNetError: 227490:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6844.218455] LNet: Removed LNI 192.168.201.145@tcp [ 6845.068595] Key type .llcrypt unregistered [ 6845.070199] Key type ._llcrypt unregistered [ 6868.950244] Key type ._llcrypt registered [ 6868.951984] Key type .llcrypt registered [ 6869.043540] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 6869.936241] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6870.132688] alg: No test for adler32 (adler32-zlib) [ 6871.134902] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [ 6871.301669] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [ 6872.976497] Key type lgssc registered [ 6874.072493] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6881.089818] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 6881.118311] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6882.546867] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6886.921991] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 6893.026348] LustreError: 228921:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6893.046420] LustreError: 228921:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 6895.242696] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6895.621816] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6900.584238] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 6908.261272] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6914.546842] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6921.833699] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6922.290092] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6927.413344] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 6928.695371] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 6936.671240] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6946.163561] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6948.288819] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6948.316719] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6948.324895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6948.332918] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6948.337410] LustreError: 228941:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6951.908022] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6951.918863] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6951.942406] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6952.938432] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6952.951846] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6955.078202] Lustre: lustre-OST0000: Not available for connect from 192.168.201.45@tcp (stopping) [ 6955.093846] Lustre: Skipped 1 previous similar message [ 6957.057366] Lustre: server umount lustre-OST0000 complete [ 6968.360157] LustreError: 228941:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6990.943689] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6991.214854] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6992.235439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6992.243682] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6997.678115] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 7002.510971] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7002.529142] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7005.737567] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7007.094340] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7012.267439] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9f3d09a25800.ost_server_uuid 50 [ 7013.601800] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9f3d09a25800.ost_server_uuid in FULL state after 0 sec [ 7017.953604] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7017.957354] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7017.974634] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7017.987485] Lustre: Skipped 1 previous similar message [ 7018.007410] Lustre: Skipped 1 previous similar message [ 7023.074035] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7023.088539] Lustre: Skipped 1 previous similar message [ 7024.437709] Lustre: server umount lustre-OST0000 complete [ 7033.330470] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7033.334402] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7033.356220] Lustre: Skipped 1 previous similar message [ 7033.384722] Lustre: Skipped 1 previous similar message [ 7036.428874] Lustre: server umount lustre-MDT0000 complete [ 7038.443379] LustreError: 228918:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7038.461831] LustreError: 228918:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 7041.673466] LustreError: 228902:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790741386 with bad export cookie 3313629077738707232 [ 7041.684844] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7041.698679] LustreError: 228902:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7042.117272] Lustre: server umount lustre-MDT0001 complete [ 7056.218738] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [ 7059.034993] Key type lgssc unregistered [ 7059.361724] LNet: 234127:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7059.388476] LNetError: 234127:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7059.397652] LNet: Removed LNI 192.168.201.145@tcp [ 7060.264593] Key type .llcrypt unregistered [ 7060.266861] Key type ._llcrypt unregistered [ 7092.723831] Key type ._llcrypt registered [ 7092.726616] Key type .llcrypt registered [ 7092.839668] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 7094.474964] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7094.571964] alg: No test for adler32 (adler32-zlib) [ 7095.820542] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [ 7096.151620] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [ 7097.896309] Key type lgssc registered [ 7099.723742] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7110.899778] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 7110.957781] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7112.652282] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7117.728572] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 7122.926533] LustreError: 235873:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7122.943055] LustreError: 235873:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 7127.121987] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7127.535838] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7132.237272] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 7134.920050] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7140.733044] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7141.132574] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7147.612499] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 7157.131557] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7157.362386] Lustre: lustre-OST0001: new disk, initializing [ 7157.372914] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7157.380514] Lustre: 238181:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7157.587735] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7157.631328] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7164.130749] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7164.136652] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7164.232678] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7164.624865] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 7176.774951] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7185.549361] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7200.236162] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7200.257465] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7200.265298] Lustre: Skipped 2 previous similar messages [ 7200.293261] Lustre: Skipped 3 previous similar messages [ 7205.346945] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7205.360878] Lustre: Skipped 3 previous similar messages [ 7206.276381] Lustre: server umount lustre-MDT0000 complete [ 7210.481251] LustreError: 235872:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7210.521438] LustreError: 235872:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 7210.539165] LustreError: 235855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790741555 with bad export cookie 4680674948579334316 [ 7210.545938] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7210.574195] LustreError: 235855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7211.229718] Lustre: server umount lustre-MDT0001 complete [ 7225.925850] Lustre: server umount lustre-OST0000 complete [ 7240.434441] Lustre: server umount lustre-OST0001 complete [ 7268.660307] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 7280.608445] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7281.126259] LustreError: 241292:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7281.270978] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7286.261445] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 7286.753804] LustreError: 241293:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7291.873647] LustreError: 241292:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7295.595144] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7296.062738] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7301.028820] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 7304.517212] Lustre: 242436:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7311.747121] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7312.109743] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7319.509990] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 7320.310215] LustreError: 242790:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7326.729725] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 7329.437041] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7338.906766] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 7347.735016] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7351.121377] Lustre: 244308:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7365.607073] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7365.608581] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7365.621533] Lustre: Skipped 3 previous similar messages [ 7365.639436] Lustre: Skipped 2 previous similar messages [ 7369.442579] Lustre: server umount mylustre-MDT0000 complete [ 7370.732794] LustreError: 241293:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7370.769483] LustreError: 241293:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [ 7373.121205] LustreError: 241269:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790741718 with bad export cookie 4680674948579337123 [ 7373.123085] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7373.129664] LustreError: 241269:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7373.599442] Lustre: server umount mylustre-MDT0001 complete [ 7387.648376] Lustre: server umount mylustre-OST0000 complete [ 7391.725215] Lustre: 235125:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790741720/real 1790741720] req@ffff8c12b7548380 x1877728497396224/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790741736 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7391.785680] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7391.799071] Lustre: Skipped 1 previous similar message [ 7392.063227] Lustre: server umount mylustre-OST0001 complete [ 7415.526987] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 7426.264602] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7426.684271] LustreError: 246616:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7426.778846] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7426.783595] Lustre: Skipped 1 previous similar message [ 7431.911189] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 7441.912781] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7447.295646] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 7450.204364] Lustre: 247756:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7457.044722] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7463.860151] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 7471.607700] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 7472.629330] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7478.794101] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 7486.015767] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7489.323497] Lustre: 249628:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7498.722335] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7498.727574] Lustre: tfs-MDT0000-lwp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7498.734764] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7498.750236] Lustre: Skipped 3 previous similar messages [ 7503.851280] LustreError: 246615:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7503.871196] LustreError: 246615:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 9 previous similar messages [ 7504.225288] Lustre: server umount tfs-MDT0000 complete [ 7508.035275] LustreError: 246592:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790741852 with bad export cookie 4680674948579338782 [ 7508.044607] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7508.482360] Lustre: server umount tfs-MDT0001 complete [ 7522.205742] Lustre: server umount tfs-OST0000 complete [ 7524.320210] Lustre: 235127:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790741853/real 1790741853] req@ffff8c11a4f6d180 x1877728497444224/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790741869 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7524.345051] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7525.864928] Lustre: server umount tfs-OST0001 complete [ 7546.470035] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 7556.983170] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7557.477147] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7557.482524] Lustre: Skipped 3 previous similar messages [ 7562.229664] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 7571.166175] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7576.559420] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 7580.384096] Lustre: 253021:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7587.398724] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7593.511907] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 7601.606312] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7602.866031] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 7607.107694] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 7614.708576] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7618.045856] Lustre: 254889:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7625.471506] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 00:19:28 (1790741968) [ 7643.105440] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7643.114854] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7643.133754] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7643.145660] Lustre: Skipped 3 previous similar messages [ 7647.393856] Lustre: server umount lustre-MDT0000 complete [ 7649.257763] LustreError: 251875:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7649.281553] LustreError: 251875:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 7 previous similar messages [ 7651.318209] LustreError: 251855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790741996 with bad export cookie 4680674948579340231 [ 7651.335384] LustreError: 251855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7651.355458] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7654.375585] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7654.383332] Lustre: Skipped 5 previous similar messages [ 7657.762295] Lustre: server umount lustre-MDT0001 complete [ 7676.385838] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7676.558326] Lustre: server umount lustre-OST0000 complete [ 7696.357365] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7696.562305] Lustre: server umount lustre-OST0001 complete [ 7704.039189] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_hostid [ 7714.727148] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 7756.346283] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7756.779889] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7756.831292] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7756.978292] Lustre: lustre-MDT0000: new disk, initializing [ 7757.167563] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7757.183427] Lustre: Skipped 3 previous similar messages [ 7757.237155] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7761.944724] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 7774.000974] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7774.146264] Lustre: 258077:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7774.225150] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7774.234195] Lustre: Skipped 1 previous similar message [ 7774.350215] Lustre: lustre-MDT0001: new disk, initializing [ 7774.534835] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7774.545375] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7780.424267] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 7793.910116] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7794.302763] Lustre: lustre-OST0000: new disk, initializing [ 7794.306744] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7794.311792] Lustre: 259663:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7796.296920] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7796.315368] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7796.440189] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7802.672966] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 7817.275635] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7817.613139] Lustre: lustre-OST0001: new disk, initializing [ 7817.616377] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7817.631367] Lustre: 260538:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7818.666879] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7818.679234] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7818.766377] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7826.242224] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 7838.965836] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 00:23:01 (1790742181) [ 7855.083372] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 00:23:18 (1790742198) [ 7873.505520] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7873.521043] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7873.527745] LustreError: Skipped 1 previous similar message [ 7873.536715] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7873.544405] Lustre: Skipped 6 previous similar messages [ 7873.576851] Lustre: Skipped 3 previous similar messages [ 7876.406683] Lustre: server umount lustre-OST0000 complete [ 7882.211730] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7886.871822] Lustre: server umount lustre-MDT0000 complete [ 7890.606183] LustreError: 258071:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790742235 with bad export cookie 4680674948579341561 [ 7890.615199] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7890.618339] LustreError: 258071:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 7897.324551] Lustre: server umount lustre-MDT0001 complete [ 7909.145787] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [ 7984.313455] Lustre: server umount lustre-OST0001 complete [ 7992.547500] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_hostid [ 8011.476541] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 8052.981576] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8053.246714] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8053.271407] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8053.346894] Lustre: lustre-MDT0000: new disk, initializing [ 8053.415204] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8053.420677] Lustre: Skipped 3 previous similar messages [ 8053.437533] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8058.234811] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 8071.141198] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8071.259200] Lustre: 265396:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8071.398658] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8071.405625] Lustre: Skipped 1 previous similar message [ 8071.542316] Lustre: lustre-MDT0001: new disk, initializing [ 8071.811196] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8071.843364] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8077.751920] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 8087.999310] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8094.419809] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8101.537804] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8102.051923] Lustre: 267366:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8103.695849] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8103.919633] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8111.754282] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 8124.947723] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8134.638173] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8134.649033] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8134.655738] Lustre: Skipped 5 previous similar messages [ 8134.657395] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8134.675051] Lustre: Skipped 8 previous similar messages [ 8138.415915] Lustre: server umount lustre-OST0000 complete [ 8143.330922] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8148.485372] Lustre: server umount lustre-MDT0000 complete [ 8149.986191] LustreError: 265403:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8150.001856] LustreError: 265403:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 13 previous similar messages [ 8153.083844] LustreError: 265386:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790742497 with bad export cookie 4680674948579343024 [ 8153.097617] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8153.099181] LustreError: 265386:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8153.529638] Lustre: server umount lustre-MDT0001 complete [ 8165.672962] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [ 8168.414756] Key type lgssc unregistered [ 8168.726463] LNet: 269126:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8168.733386] LNetError: 269126:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8168.763362] LNet: Removed LNI 192.168.201.145@tcp [ 8169.922146] Key type .llcrypt unregistered [ 8169.926888] Key type ._llcrypt unregistered [ 8200.870651] Key type ._llcrypt registered [ 8200.872431] Key type .llcrypt registered [ 8200.997944] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 8202.236118] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8202.334476] alg: No test for adler32 (adler32-zlib) [ 8203.350987] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [ 8203.547068] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [ 8205.168417] Key type lgssc registered [ 8206.637753] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8215.658566] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 8215.683493] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8217.271148] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8221.633479] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 8227.811733] LustreError: 270605:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8227.843966] LustreError: 270605:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 8231.106464] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8231.354405] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8235.732419] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 8244.366954] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8250.494296] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8257.830802] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8258.348885] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8262.449798] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8265.195844] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 8273.242183] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8274.220669] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8277.352594] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8277.356181] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8277.388040] Lustre: Skipped 1 previous similar message [ 8282.684439] Lustre: server umount lustre-OST0000 complete [ 8287.714372] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8287.717713] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8287.731772] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8287.750155] Lustre: Skipped 1 previous similar message [ 8287.769875] Lustre: Skipped 1 previous similar message [ 8292.834438] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8292.851658] Lustre: Skipped 1 previous similar message [ 8293.254643] Lustre: server umount lustre-MDT0000 complete [ 8297.299810] LustreError: 270586:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790742642 with bad export cookie 319067183952187608 [ 8297.302361] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8297.315192] LustreError: 270586:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8297.759922] Lustre: server umount lustre-MDT0001 complete [ 8312.451894] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [ 8315.192158] Key type lgssc unregistered [ 8315.620467] LNet: 274189:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8315.635253] LNetError: 274189:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8315.670690] LNet: Removed LNI 192.168.201.145@tcp [ 8316.844264] Key type .llcrypt unregistered [ 8316.846423] Key type ._llcrypt unregistered [ 8336.478985] Key type ._llcrypt registered [ 8336.483834] Key type .llcrypt registered [ 8336.636781] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 8337.740597] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8337.977822] alg: No test for adler32 (adler32-zlib) [ 8339.202777] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [ 8339.452731] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [ 8341.120568] Key type lgssc registered [ 8342.624236] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8352.191462] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 8352.236477] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8367.840507] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8372.960842] LustreError: 275399:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.145@tcp: failed processing log, type 4: rc = -110 [ 8399.777097] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8406.652818] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 8413.338660] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8415.593570] Lustre: server umount lustre-OST0000 complete [ 8432.295429] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [ 8435.474199] Key type lgssc unregistered [ 8435.887807] LNet: 277029:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8435.900287] LNetError: 277029:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8435.928146] LNet: Removed LNI 192.168.201.145@tcp [ 8437.031654] Key type .llcrypt unregistered [ 8437.039987] Key type ._llcrypt unregistered [ 8487.301689] Key type ._llcrypt registered [ 8487.309533] Key type .llcrypt registered [ 8487.379996] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 8488.163026] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8488.291968] alg: No test for adler32 (adler32-zlib) [ 8489.446099] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [ 8489.910392] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [ 8491.664285] Key type lgssc registered [ 8492.977038] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8507.545692] loop0: detected capacity change from 0 to 204800000 [ 8507.882923] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8507.894980] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8508.264737] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8509.602838] loop0: detected capacity change from 0 to 204800000 [ 8509.824347] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8509.837746] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8510.251194] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8511.548349] loop0: detected capacity change from 0 to 204800000 [ 8511.936067] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8511.970233] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8512.251665] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8513.982492] loop0: detected capacity change from 0 to 204800000 [ 8514.229030] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8514.256943] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8514.406298] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8515.669087] loop0: detected capacity change from 0 to 204800000 [ 8515.715015] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8520.866255] loop0: detected capacity change from 0 to 204800000 [ 8520.917036] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8526.507494] loop0: detected capacity change from 0 to 204800000 [ 8526.541259] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8531.631028] loop0: detected capacity change from 0 to 204800000 [ 8531.661503] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8536.896528] loop0: detected capacity change from 0 to 204800000 [ 8537.051428] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 8537.112173] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8537.294282] Lustre: 280461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8537.311771] Lustre: 280461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8537.342556] Lustre: 280461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8537.371219] Lustre: 280461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8537.418067] Lustre: 280461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8537.436391] Lustre: 280461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8537.458920] Lustre: 280461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8537.477607] Lustre: 280461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8537.502374] Lustre: 280461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8537.519253] Lustre: 280461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8537.538036] Lustre: 280461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8537.557817] Lustre: 280461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8537.580567] Lustre: 280461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8537.607509] Lustre: 280461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8544.646688] Lustre: server umount MGS complete [ 8545.824555] loop0: detected capacity change from 0 to 204800000 [ 8545.926500] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8547.055156] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8547.130514] Lustre: lustre-MDT0000: Aborting client recovery [ 8548.353432] loop1: detected capacity change from 0 to 204800000 [ 8548.514366] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8548.686469] Lustre: 280896:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8548.718791] Lustre: 280896:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8548.739960] Lustre: 280896:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8548.773945] Lustre: 280896:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8548.828408] Lustre: 280896:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8548.858933] Lustre: 280896:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8548.887795] Lustre: 280896:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8548.935749] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8549.280310] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8549.382504] Lustre: lustre-MDT0001: Aborting client recovery [ 8550.355100] loop2: detected capacity change from 0 to 204800000 [ 8550.431074] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8550.462664] Lustre: 280976:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8550.467608] Lustre: 280976:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8550.473846] Lustre: 280976:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8550.482437] Lustre: 280976:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8550.486301] Lustre: 280976:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8550.490386] Lustre: 280976:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8550.494862] Lustre: 280976:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8550.549069] Lustre: Found index 0 for lustre-OST0000, updating log [ 8550.563625] Lustre: 280995:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8550.610239] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8550.657761] Lustre: lustre-OST0000: Aborting recovery [ 8551.620620] loop3: detected capacity change from 0 to 204800000 [ 8551.709058] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8551.788627] Lustre: 281060:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8551.795628] Lustre: 281060:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8551.802557] Lustre: 281060:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8551.810647] Lustre: 281060:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8551.816978] Lustre: 281060:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8551.821953] Lustre: 281060:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8551.829511] Lustre: 281060:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8551.844629] Lustre: Found index 1 for lustre-OST0001, updating log [ 8551.897834] Lustre: 281066:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8551.991772] Lustre: lustre-OST0001: Aborting recovery [ 8557.123336] LustreError: 280839:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 11, retries 0, failed: rc = -108 [ 8558.056662] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8558.129376] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8558.208694] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8558.211719] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8560.115466] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8560.117493] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8561.470137] LustreError: 280806:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 2183us: rc = -2 [ 8562.050158] LustreError: 280807:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 613us: rc = -2 [ 8562.061928] LustreError: 280807:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 8574.950518] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8574.955180] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8574.971734] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8575.458643] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8575.476859] Lustre: Skipped 2 previous similar messages [ 8575.488695] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8575.496516] Lustre: Skipped 3 previous similar messages [ 8580.582424] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8580.585328] Lustre: Skipped 3 previous similar messages [ 8581.276452] Lustre: server umount lustre-MDT0000 complete [ 8582.419437] LustreError: 280789:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790742927 with bad export cookie 15251884204568902684 [ 8582.421379] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8582.436548] LustreError: 280789:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8585.698327] LustreError: 280805:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8585.698962] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8585.729423] LustreError: 280805:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 8585.741600] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8587.746522] LustreError: 280803:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8587.769187] LustreError: 280803:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 8589.105498] Lustre: server umount lustre-MDT0001 complete [ 8596.677896] Lustre: server umount lustre-OST0000 complete [ 8612.320189] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8612.758322] Lustre: server umount lustre-OST0001 complete [ 8621.844765] Lustre: DEBUG MARKER: == conf-sanity test 108c: mkfs.lustre --mmp option enables MMP feature ========================================================== 00:36:04 (1790742964) [ 8647.208703] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 8689.389468] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8689.731177] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8689.768599] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8689.915925] Lustre: lustre-MDT0000: new disk, initializing [ 8690.023334] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8690.030489] Lustre: Skipped 1 previous similar message [ 8690.048173] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8694.655314] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 8701.945664] Lustre: server umount lustre-MDT0000 complete [ 8719.655274] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 00:37:42 (1790743062) [ 8749.638638] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_hostid [ 8760.412576] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 8798.713920] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8799.081675] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8799.121770] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8799.125887] Lustre: Skipped 1 previous similar message [ 8799.273960] Lustre: lustre-MDT0000: new disk, initializing [ 8799.448315] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8799.490662] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8805.003795] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 8816.798230] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8816.957031] Lustre: 288175:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8817.054679] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8817.067937] Lustre: Skipped 1 previous similar message [ 8817.228856] Lustre: lustre-MDT0001: new disk, initializing [ 8817.394438] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8817.463912] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8817.469956] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8822.488698] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 8832.841951] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8839.419459] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8846.913127] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8847.166286] Lustre: lustre-OST0000: new disk, initializing [ 8847.169926] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8847.176148] Lustre: 290147:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8847.312481] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8853.922345] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 8854.624859] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8854.641502] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8854.767937] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8863.855667] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8869.331503] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8880.738968] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8888.973331] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8936.936651] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8936.942796] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8936.951294] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8936.970571] Lustre: Skipped 2 previous similar messages [ 8936.989421] Lustre: Skipped 4 previous similar messages [ 8947.172521] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8947.182039] Lustre: Skipped 3 previous similar messages [ 8948.192184] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8948.529251] Lustre: server umount lustre-OST0000 complete [ 8955.875043] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8955.886849] LustreError: Skipped 1 previous similar message [ 8955.896168] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8958.675178] LustreError: 292276:0:(ldlm_resource.c:1227:ldlm_resource_complain()) MGS: namespace resource [0x65727473756c:0x5:0x0].0x0 (00000000ff00acfe) refcount nonzero (2) after lock cleanup; forcing cleanup. [ 8958.745880] Lustre: server umount lustre-MDT0000 complete [ 8962.361919] LustreError: 288970:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790743307 with bad export cookie 15251884204568905652 [ 8962.369743] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8962.373522] LustreError: 288970:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8962.842931] Lustre: server umount lustre-MDT0001 complete [ 8971.756278] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8976.381799] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 8980.882311] Lustre: server umount MGS complete [ 8991.345361] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8991.707441] LustreError: 294056:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8991.733662] LustreError: 294069:0:(qmt_pool.c:1648:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8991.786026] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8995.788893] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 8996.837123] LustreError: 294057:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9001.963448] LustreError: 294056:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9004.251983] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9009.527907] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 9018.649594] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9025.840119] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9032.630628] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9039.322716] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 9046.645182] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9078.270268] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9078.284437] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9078.298163] Lustre: Skipped 1 previous similar message [ 9078.308331] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9078.316064] Lustre: Skipped 3 previous similar messages [ 9082.345690] Lustre: server umount lustre-OST0000 complete [ 9086.434150] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9086.442750] LustreError: Skipped 1 previous similar message [ 9092.682428] Lustre: server umount lustre-MDT0000 complete [ 9095.655522] LustreError: 294786:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9095.695618] LustreError: 294786:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9097.214271] LustreError: 296782:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790743442 with bad export cookie 15251884204568907493 [ 9097.221082] LustreError: 296782:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9097.223241] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9097.847551] Lustre: server umount lustre-MDT0001 complete [ 9112.819241] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [ 9117.106294] Key type lgssc unregistered [ 9117.829167] LNet: 298002:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9117.851857] LNetError: 298002:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9117.889574] LNet: Removed LNI 192.168.201.145@tcp [ 9119.404635] Key type .llcrypt unregistered [ 9119.407089] Key type ._llcrypt unregistered [ 9157.662703] Key type ._llcrypt registered [ 9157.667265] Key type .llcrypt registered [ 9157.776842] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_hostid [ 9176.314115] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 9177.595847] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9177.738293] alg: No test for adler32 (adler32-zlib) [ 9179.130654] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [ 9179.556258] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [ 9181.377874] Key type lgssc registered [ 9182.757107] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9220.722655] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 9220.784795] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9222.105538] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9222.131377] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9222.194112] Lustre: lustre-MDT0000: new disk, initializing [ 9222.360918] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9222.397770] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9227.431571] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 9238.756845] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9238.873892] Lustre: 301981:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9238.929288] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9238.933594] Lustre: Skipped 1 previous similar message [ 9239.045793] Lustre: lustre-MDT0001: new disk, initializing [ 9239.208077] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9239.261070] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9239.273571] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9243.434273] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 9253.449218] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9260.840540] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9268.974915] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9269.588452] Lustre: lustre-OST0000: new disk, initializing [ 9269.591855] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9269.599619] Lustre: 303946:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9269.805482] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9275.984936] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9276.000246] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9276.148684] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9279.015480] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 9289.535149] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9296.777751] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9311.035725] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9315.978235] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9324.378063] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9373.668138] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9373.686646] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9373.687480] LustreError: Skipped 1 previous similar message [ 9373.689887] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9373.701706] Lustre: Skipped 1 previous similar message [ 9375.124903] Lustre: server umount lustre-OST0000 complete [ 9382.372868] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9382.395776] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9382.429708] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9382.436064] Lustre: Skipped 1 previous similar message [ 9383.920216] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9383.923144] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9383.938428] Lustre: Skipped 1 previous similar message [ 9385.812444] Lustre: server umount lustre-MDT0000 complete [ 9390.868898] LustreError: 301972:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790743735 with bad export cookie 4680546075534470201 [ 9390.905602] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9391.341494] Lustre: server umount lustre-MDT0001 complete [ 9402.676526] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9407.906476] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 9414.110733] Lustre: server umount MGS complete [ 9428.169816] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9428.824902] LustreError: 307775:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9428.839473] LustreError: 307775:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 9428.852765] LustreError: 307788:0:(qmt_pool.c:1648:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9428.926784] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9434.085942] LustreError: 307776:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9434.418920] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 9438.197439] LustreError: 307775:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9443.303858] LustreError: 307776:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9444.399095] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9444.798660] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9449.641907] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 9457.037608] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9463.152522] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9470.643286] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9471.118801] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9477.854981] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 9485.347272] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9515.491208] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9515.496149] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9515.508510] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9520.616361] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9520.619100] Lustre: Skipped 1 previous similar message [ 9520.931432] Lustre: server umount lustre-OST0000 complete [ 9525.731650] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9525.742477] Lustre: Skipped 2 previous similar messages [ 9530.849245] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9530.859778] Lustre: Skipped 4 previous similar messages [ 9531.349461] Lustre: server umount lustre-MDT0000 complete [ 9534.925442] LustreError: 307756:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790743879 with bad export cookie 4680546075534472154 [ 9534.935864] LustreError: 307756:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9534.948796] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9535.310506] Lustre: server umount lustre-MDT0001 complete [ 9547.696821] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [ 9550.724622] Key type lgssc unregistered [ 9551.086251] LNet: 311717:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9551.111196] LNetError: 311717:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9552.170399] LNet: Removed LNI 192.168.201.145@tcp [ 9552.968124] Key type .llcrypt unregistered [ 9552.969389] Key type ._llcrypt unregistered [ 9573.112934] Key type ._llcrypt registered [ 9573.115733] Key type .llcrypt registered [ 9573.221324] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_hostid [ 9586.391879] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 9587.679502] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9587.782688] alg: No test for adler32 (adler32-zlib) [ 9588.941211] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [ 9589.253362] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [ 9590.944421] Key type lgssc registered [ 9592.314046] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9616.930160] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [ 9630.870350] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 9630.917961] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9632.142147] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9632.165132] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9632.250392] Lustre: lustre-MDT0000: new disk, initializing [ 9632.379509] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9632.402107] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9636.611382] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [ 9647.449614] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9647.889469] Lustre: lustre-OST0000: new disk, initializing [ 9647.896388] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9647.899088] Lustre: Skipped 1 previous similar message [ 9647.909471] Lustre: 315764:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9647.978821] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9649.318888] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9649.326409] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9649.366254] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9655.267351] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [10407.394388] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10407.407679] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10411.592731] Lustre: server umount lustre-MDT0000 complete [10415.474591] LustreError: 314951:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790744760 with bad export cookie 2780113055516063545 [10415.478435] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10415.509179] LustreError: 314951:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10415.940398] Lustre: server umount lustre-OST0000 complete [10445.767420] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_hostid [10456.165288] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [10487.752131] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10488.027864] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10488.084520] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10488.153137] Lustre: lustre-MDT0000: new disk, initializing [10488.230252] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10488.245901] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10492.469817] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [10503.686023] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10503.845777] Lustre: 320094:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10503.896265] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10503.898990] Lustre: Skipped 1 previous similar message [10503.959946] Lustre: lustre-MDT0001: new disk, initializing [10504.091417] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10504.157826] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10504.175600] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10509.410872] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [10520.372561] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10526.283340] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10532.458308] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10532.804925] Lustre: lustre-OST0000: new disk, initializing [10532.808895] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10532.834644] Lustre: 322062:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10532.993939] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10539.433324] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [10542.110969] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10542.119730] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10542.217456] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10550.238565] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10555.056355] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10555.287725] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10559.784836] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10559.992735] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10562.547781] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10562.564701] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10562.588672] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10563.051779] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10568.176557] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10568.181740] Lustre: Skipped 2 previous similar messages [10573.284500] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10573.289774] Lustre: Skipped 1 previous similar message [10576.353347] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10576.509850] Lustre: server umount lustre-OST0000 complete [10580.962847] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10580.969575] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10580.989073] Lustre: Skipped 1 previous similar message [10580.994472] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10580.998626] Lustre: Skipped 1 previous similar message [10583.529692] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10586.744932] Lustre: server umount lustre-MDT0000 complete [10590.575674] LustreError: 320087:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790744935 with bad export cookie 2780113055519004364 [10590.578471] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10590.589166] LustreError: 320087:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10591.049466] Lustre: server umount lustre-MDT0001 complete [10594.596933] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10596.768248] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 01:08:59 (1790744939) [10603.191150] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10603.584195] LustreError: 324375:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10603.599695] LustreError: 324375:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [10603.638234] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10608.179295] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [10609.127203] LustreError: 324376:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10613.228358] LustreError: 324375:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10617.852490] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10618.277980] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10622.484285] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [10629.618928] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10635.891746] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10644.340370] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10645.003195] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10656.673891] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [10666.713987] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10673.798268] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10673.951355] Lustre: lustre-OST0001: new disk, initializing [10673.957981] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10673.967174] Lustre: 327213:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10674.102546] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10680.034635] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [10681.928615] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10681.938695] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10681.991553] LustreError: 327673:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10681.999838] LustreError: 327673:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10683.050402] LustreError: 327673:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10683.066805] LustreError: 327673:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10684.067368] LustreError: 327677:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10686.147108] LustreError: 327677:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10686.163305] LustreError: 327677:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10689.911760] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10690.172960] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10690.284805] LustreError: 327677:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10690.308091] LustreError: 327677:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [10695.022469] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9f3d034e9000.ost_server_uuid 50 [10696.175021] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9f3d034e9000.ost_server_uuid in FULL state after 0 sec [10698.544571] LustreError: 327673:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10698.566334] LustreError: 327673:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [10700.826494] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9f3d034e9000.ost_server_uuid 50 [10701.902261] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9f3d034e9000.ost_server_uuid in FULL state after 0 sec [10703.752782] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10718.182178] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10718.194290] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10718.220259] Lustre: Skipped 1 previous similar message [10718.239948] Lustre: Skipped 3 previous similar messages [10722.857614] Lustre: server umount lustre-OST0001 complete [10723.305229] LustreError: 326199:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10723.337209] LustreError: 326199:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10728.421395] LustreError: 326199:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10728.434978] LustreError: 326199:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [10729.960570] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10729.970809] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10735.463966] Lustre: server umount lustre-OST0000 complete [10741.221476] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10741.226793] LustreError: Skipped 1 previous similar message [10741.236772] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10741.244960] Lustre: Skipped 5 previous similar messages [10745.472955] Lustre: server umount lustre-MDT0000 complete [10748.853192] LustreError: 325111:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790745093 with bad export cookie 2780113055519005246 [10748.858877] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10748.865977] LustreError: 325111:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10749.211848] Lustre: server umount lustre-MDT0001 complete [10760.843529] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [10763.579210] Key type lgssc unregistered [10763.845874] LNet: 329897:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10763.854974] LNetError: 329897:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10763.869802] LNet: Removed LNI 192.168.201.145@tcp [10764.769934] Key type .llcrypt unregistered [10764.772649] Key type ._llcrypt unregistered [10787.397046] Key type ._llcrypt registered [10787.398988] Key type .llcrypt registered [10787.493528] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [10788.544338] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10788.580074] alg: No test for adler32 (adler32-zlib) [10789.793243] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [10790.063251] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [10791.769726] Key type lgssc registered [10793.150172] Lustre: Echo OBD driver; http://www.lustre.org/ [10800.720377] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [10800.734472] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10802.266142] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10807.098812] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [10812.396683] LustreError: 331336:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10812.435823] LustreError: 331336:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [10816.112063] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10817.521159] LustreError: 331335:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10822.631851] LustreError: 331336:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10824.580868] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10825.079314] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10830.921305] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [10842.435248] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10843.015378] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10849.133145] LustreError: 333016:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10849.173503] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10851.106360] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [10854.393669] LustreError: 333284:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10854.418704] LustreError: 333284:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [10859.502626] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10864.616627] LustreError: 333284:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10864.645703] LustreError: 333284:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [10866.247928] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10866.460371] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10871.838056] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10873.061941] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [10880.328977] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10886.997823] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10887.189366] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10989.544639] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10989.554602] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10989.570158] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10989.579957] Lustre: Skipped 1 previous similar message [10992.558765] Lustre: server umount lustre-OST0001 complete [10994.663189] LustreError: 333493:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10994.679599] LustreError: 333493:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10999.784932] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10999.791793] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10999.808619] Lustre: Skipped 2 previous similar messages [10999.843558] Lustre: Skipped 1 previous similar message [11004.645375] Lustre: server umount lustre-OST0000 complete [11009.507409] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11009.524169] LustreError: Skipped 1 previous similar message [11009.537598] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11009.583486] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11015.141320] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11015.159717] Lustre: Skipped 3 previous similar messages [11019.235106] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11019.250259] Lustre: Skipped 1 previous similar message [11023.328121] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11023.500971] Lustre: server umount lustre-MDT0000 complete [11027.251039] LustreError: 331317:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790745372 with bad export cookie 80259891103919570 [11027.255069] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11027.261824] LustreError: 331317:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11027.608909] Lustre: server umount lustre-MDT0001 complete [11038.437873] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [11041.371483] Key type lgssc unregistered [11041.704694] LNet: 336279:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11041.727679] LNetError: 336279:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11041.757643] LNet: Removed LNI 192.168.201.145@tcp [11042.805610] Key type .llcrypt unregistered [11042.807517] Key type ._llcrypt unregistered [11085.076940] Key type ._llcrypt registered [11085.079938] Key type .llcrypt registered [11085.198453] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [11086.405948] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11086.503846] alg: No test for adler32 (adler32-zlib) [11087.734469] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [11088.084134] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [11089.776293] Key type lgssc registered [11091.292236] Lustre: Echo OBD driver; http://www.lustre.org/ [11099.370301] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [11099.403623] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11100.859382] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11104.991574] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [11110.376281] LustreError: 338584:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11110.396358] LustreError: 338584:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11112.943170] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11113.357354] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11117.335939] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [11119.348889] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11125.291121] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11125.620798] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11126.650862] LustreError: 340070:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11126.663481] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11131.170831] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [11131.883730] LustreError: 340070:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11137.005950] LustreError: 340072:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11139.512629] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11139.668965] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11144.715524] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11145.907599] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [11155.216801] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11161.787250] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11174.903525] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11174.908298] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11174.936802] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11180.515687] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11180.529612] Lustre: Skipped 7 previous similar messages [11181.260145] Lustre: server umount lustre-MDT0000 complete [11185.502396] LustreError: 338563:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790745530 with bad export cookie 3663441140962585801 [11185.507135] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11185.514174] LustreError: 338563:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11185.634660] LustreError: 338578:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11185.636358] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11185.669251] LustreError: 338578:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [11185.671215] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11185.719569] Lustre: Skipped 4 previous similar messages [11186.235595] Lustre: server umount lustre-MDT0001 complete [11192.928662] Lustre: server umount lustre-OST0000 complete [11197.741383] Lustre: server umount lustre-OST0001 complete [11215.246766] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [11223.942718] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11224.306124] LustreError: 343845:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11224.370188] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11227.873950] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [11235.775950] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11236.065631] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11240.438669] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [11243.382376] Lustre: 344985:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11249.227256] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11250.645150] LustreError: 345337:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11250.670254] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11250.681488] LustreError: 345337:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11256.577829] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [11266.215800] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11266.668749] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11266.682276] Lustre: Skipped 1 previous similar message [11271.689456] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11275.009467] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [11282.638602] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11286.448878] Lustre: 346856:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11292.132490] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11292.149454] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11292.162160] Lustre: Skipped 3 previous similar messages [11292.192591] Lustre: Skipped 4 previous similar messages [11297.252453] LustreError: 343842:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11297.282393] LustreError: 343842:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 8 previous similar messages [11299.077069] Lustre: server umount lustre-MDT0000 complete [11302.720706] LustreError: 343828:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790745647 with bad export cookie 3663441140962587502 [11302.737419] LustreError: 343828:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11302.743154] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11302.989899] Lustre: server umount lustre-MDT0001 complete [11316.967647] Lustre: server umount lustre-OST0000 complete [11330.930190] Lustre: server umount lustre-OST0001 complete [11335.467995] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11336.747896] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11338.403633] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 01:21:21 (1790745681) [11362.841812] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [11373.503340] loop0: detected capacity change from 0 to 1099511627776 [11374.491078] SGI XFS with ACLs, security attributes, quota, no debug enabled [11374.501284] *** VALIDATE xfs *** [11374.525467] XFS (loop0): Mounting V5 Filesystem [11374.558809] XFS (loop0): Ending clean mount [11376.139377] loop1: detected capacity change from 0 to 18691697672192 [11496.667743] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_hostid [11510.504273] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [11550.434553] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11550.604172] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11550.622197] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11550.673839] Lustre: lustre-MDT0000: new disk, initializing [11550.723692] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11550.736161] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11555.933212] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [11567.705913] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11567.891726] Lustre: 351992:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11568.030693] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11568.036303] Lustre: Skipped 1 previous similar message [11568.116413] Lustre: lustre-MDT0001: new disk, initializing [11568.271826] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11568.295036] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11572.912622] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [11581.464558] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11586.417541] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11594.842092] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11595.241349] Lustre: lustre-OST0000: new disk, initializing [11595.246162] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11595.258298] Lustre: 353961:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11596.582972] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11596.592490] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11596.690419] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11601.946810] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [11611.143478] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11615.394866] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11615.565537] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11619.966632] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11620.157733] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11622.374358] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11622.381301] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11622.383092] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11622.397386] Lustre: Skipped 1 previous similar message [11628.876585] Lustre: server umount lustre-OST0000 complete [11634.657027] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11634.660299] LustreError: Skipped 1 previous similar message [11634.668303] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11634.682479] Lustre: Skipped 1 previous similar message [11634.686407] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11634.702608] Lustre: Skipped 2 previous similar messages [11639.937758] Lustre: server umount lustre-MDT0000 complete [11642.853289] LustreError: 352004:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11642.874725] LustreError: 352004:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [11643.657092] LustreError: 351986:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790745988 with bad export cookie 3663441140962589049 [11643.667708] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11643.970880] Lustre: server umount lustre-MDT0001 complete [11654.146986] XFS (loop0): Unmounting Filesystem [11660.756071] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 01:26:43 (1790746003) [11667.929149] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11668.381301] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11668.388514] Lustre: Skipped 2 previous similar messages [11673.227902] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [11682.406018] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11688.150734] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [11697.107784] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11703.865633] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11711.416574] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11718.521343] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [11727.406211] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11737.569489] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11737.576933] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11737.593242] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11737.602328] Lustre: Skipped 2 previous similar messages [11737.620723] Lustre: Skipped 3 previous similar messages [11743.206087] Lustre: server umount lustre-OST0000 complete [11753.900663] Lustre: server umount lustre-MDT0000 complete [11757.606348] LustreError: 356501:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790746102 with bad export cookie 3663441140962589938 [11757.626088] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11758.077918] Lustre: server umount lustre-MDT0001 complete [11769.915322] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [11772.471456] Key type lgssc unregistered [11772.736238] LNet: 360148:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11772.750124] LNetError: 360148:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11772.774230] LNet: Removed LNI 192.168.201.145@tcp [11773.726161] Key type .llcrypt unregistered [11773.734537] Key type ._llcrypt unregistered [11800.360572] Key type ._llcrypt registered [11800.362930] Key type .llcrypt registered [11800.457951] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [11801.303696] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11801.325988] alg: No test for adler32 (adler32-zlib) [11802.349932] Lustre: Lustre: Build Version: 2.17.59_3_g4e55f3f [11802.595754] LNet: Added LNI 192.168.201.145@tcp [8/256/0/180] [11804.248518] Key type lgssc registered [11805.374890] Lustre: Echo OBD driver; http://www.lustre.org/ [11812.483309] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [11812.504688] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11813.889728] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11817.720902] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [11824.100147] LustreError: 361732:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11824.119442] LustreError: 361732:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11825.351381] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11825.695866] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11830.363701] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [11838.811267] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11846.213764] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11855.956374] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11856.428662] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11863.632612] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [11871.880830] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11878.476337] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.45@tcp (stopping) [11880.931209] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11880.950289] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11880.954086] Lustre: Skipped 1 previous similar message [11883.556288] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.45@tcp (stopping) [11884.333044] Lustre: server umount lustre-MDT0001 complete [11886.062177] LustreError: 361732:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11886.092424] LustreError: 361732:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11888.682565] LustreError: 363524:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.45@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11891.171170] LustreError: 361733:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11891.184626] LustreError: 361733:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11893.638323] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11893.757970] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11893.764795] Lustre: Found index 1 for lustre-MDT0001, updating log [11893.812239] Lustre: 361719:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11894.184759] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11898.356405] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [11899.375305] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11902.972035] LustreError: 365335:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11902.982052] LustreError: 365335:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11902.988788] LustreError: 361723:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11903.058973] LustreError: 365338:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.145@tcp: cfg command failed: rc = -17 [11903.068555] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11903.068555] [12204.097472] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.45@tcp (stopping) [12206.577256] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12206.594198] Lustre: Skipped 1 previous similar message [12209.196139] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.45@tcp (stopping) [12209.204763] Lustre: Skipped 2 previous similar messages [12209.442597] Lustre: server umount lustre-MDT0001 complete [12211.689658] LustreError: 361733:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12211.718401] LustreError: 361733:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [12220.030556] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12220.199826] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12220.212180] Lustre: Found index 1 for lustre-MDT0001, updating log [12220.258907] Lustre: 361719:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12220.657383] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12226.037401] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12226.155350] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [12229.626291] LustreError: 373809:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12229.632494] LustreError: 373809:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12229.641867] LustreError: 361723:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12229.654216] LustreError: 361723:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12229.718791] LustreError: 373812:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12229.725944] LustreError: 373812:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12229.734671] LustreError: 373812:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.145@tcp: cfg command failed: rc = -17 [12229.750611] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.145@tcp [12229.750611] [12533.218386] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12533.223640] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12533.245480] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12533.271391] Lustre: Skipped 2 previous similar messages [12533.290827] Lustre: Skipped 1 previous similar message [12539.016243] Lustre: server umount lustre-MDT0001 complete [12539.939451] LustreError: 361727:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.45@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12539.955905] LustreError: 361727:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [12549.773081] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12549.930425] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12549.938664] Lustre: Found index 1 for lustre-MDT0001, updating log [12550.008994] Lustre: 361720:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12550.528724] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12555.170335] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [12555.761200] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12556.285521] LustreError: 382141:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12556.296961] LustreError: 382141:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12556.311765] LustreError: 361723:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12556.323085] LustreError: 361723:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12556.419410] LustreError: 382147:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12556.434486] LustreError: 382147:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.145@tcp: cfg command failed: rc = -114 [12556.442787] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12556.442787] [12883.434865] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12883.442266] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12883.442396] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12883.452380] Lustre: Skipped 2 previous similar messages [12883.475974] Lustre: Skipped 5 previous similar messages [12885.656513] Lustre: server umount lustre-MDT0000 complete [12888.569412] LustreError: 361727:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12888.590935] LustreError: 361727:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 8 previous similar messages [12889.794386] LustreError: 361712:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790747234 with bad export cookie 14508950927151407087 [12889.809206] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12893.675054] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12896.352718] Lustre: server umount lustre-MDT0001 complete [12900.048835] Lustre: server umount lustre-OST0000 complete [12909.656880] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_hostid [12919.468583] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [12967.560693] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 01:48:30 (1790747310) [12974.750503] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12974.995150] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12975.025430] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12975.133304] Lustre: lustre-MDT0000: new disk, initializing [12975.210481] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12975.231510] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12979.151685] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [12990.360460] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12990.556514] Lustre: 392769:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12990.606111] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12990.610170] Lustre: Skipped 1 previous similar message [12990.676283] Lustre: lustre-MDT0001: new disk, initializing [12990.748955] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12990.807317] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12990.832963] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12995.805432] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [13007.300985] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13014.821535] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13022.037895] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13022.322952] Lustre: lustre-OST0000: new disk, initializing [13022.327888] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13022.334439] Lustre: 394737:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13022.428487] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13028.410594] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_default_debug -1 all [13032.523155] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13032.537265] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13032.681052] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13038.174746] Lustre: DEBUG MARKER: oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13046.764069] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [13046.773969] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13046.785939] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13046.788767] Lustre: Skipped 2 previous similar messages [13048.019258] Lustre: server umount lustre-MDT0000 complete [13048.299446] LustreError: 392782:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13048.319748] LustreError: 392782:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [13052.341521] LustreError: 392763:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790747397 with bad export cookie 14508950927151409047 [13052.343767] LustreError: 393576:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [13052.351356] LustreError: 392763:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [13052.353763] LustreError: MGC192.168.201.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13052.367916] LustreError: 393576:0:(client.c:1394:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@0000000015731c00 x1877733432219392/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [13052.731420] Lustre: server umount lustre-MDT0001 complete [13066.741192] Lustre: server umount lustre-OST0000 complete [13098.893359] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing set_hostid [13110.011883] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing load_modules_local [13142.566360] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12853 sec ========= 01:51:25 (1790747485) [13144.458825] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 01:51:27 (1790747487) === [13147.525133] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 01:51:30 (1790747490) === [13189.432349] Lustre: DEBUG MARKER: oleg145-server.virtnet: executing unload_modules_local [13192.760565] Key type lgssc unregistered [13193.327965] LNet: 401110:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13193.335580] LNetError: 401110:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13193.358196] LNet: Removed LNI 192.168.201.145@tcp [13194.473569] Key type .llcrypt unregistered [13194.475442] Key type ._llcrypt unregistered