[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 464703308 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.003116] x2apic enabled [ 0.004007] Switched APIC routing to physical x2apic. [ 0.005013] kvm-guest: setup PV IPIs [ 0.008652] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009021] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010011] pid_max: default: 32768 minimum: 301 [ 0.011122] LSM: Security Framework initializing [ 0.012069] Yama: becoming mindful. [ 0.013022] SELinux: Initializing. [ 0.014068] *** VALIDATE selinux *** [ 0.022217] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027207] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028132] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029124] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030118] *** VALIDATE tmpfs *** [ 0.031329] *** VALIDATE proc *** [ 0.032263] *** VALIDATE cgroup *** [ 0.034010] *** VALIDATE cgroup2 *** [ 0.035281] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036151] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037010] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038029] Spectre V2 : User space: Vulnerable [ 0.039009] Speculative Store Bypass: Vulnerable [ 0.041623] debug: unmapping init [mem 0xffffffff94a59000-0xffffffff94a60fff] [ 0.044131] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045724] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046023] ... version: 2 [ 0.047013] ... bit width: 48 [ 0.048012] ... generic registers: 4 [ 0.049012] ... value mask: 0000ffffffffffff [ 0.050017] ... max period: 00007fffffffffff [ 0.051013] ... fixed-purpose events: 3 [ 0.052009] ... event mask: 000000070000000f [ 0.053274] rcu: Hierarchical SRCU implementation. [ 0.055268] smp: Bringing up secondary CPUs ... [ 0.056455] x86: Booting SMP configuration: [ 0.057020] .... node #0, CPUs: #1 #2 #3 [ 0.059570] smp: Brought up 1 node, 4 CPUs [ 0.060676] smpboot: Max logical packages: 1 [ 0.061009] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.137109] node 0 deferred pages initialised in 73ms [ 0.141159] devtmpfs: initialized [ 0.142220] x86/mm: Memory block size: 128MB [ 0.144663] gcov: version magic: 0x41383552 [ 0.146117] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.147111] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.148243] pinctrl core: initialized pinctrl subsystem [ 0.149178] [ 0.149571] ************************************************************* [ 0.150026] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.151013] ** ** [ 0.152016] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.153015] ** ** [ 0.154012] ** This means that this kernel is built to expose internal ** [ 0.155015] ** IOMMU data structures, which may compromise security on ** [ 0.156017] ** your system. ** [ 0.157014] ** ** [ 0.158013] ** If you see this message and you are not debugging the ** [ 0.159014] ** kernel, report this immediately to your vendor! ** [ 0.160014] ** ** [ 0.161013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.162011] ************************************************************* [ 0.163770] NET: Registered protocol family 16 [ 0.164454] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.165067] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.166060] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.168012] cpuidle: using governor menu [ 0.170076] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.173489] PCI: Using configuration type 1 for base access [ 0.175118] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.184117] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.186016] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.188022] cryptd: max_cpu_qlen set to 1000 [ 0.190187] ACPI: Added _OSI(Module Device) [ 0.191015] ACPI: Added _OSI(Processor Device) [ 0.192014] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.193017] ACPI: Added _OSI(Processor Aggregator Device) [ 0.197566] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.205856] ACPI: Interpreter enabled [ 0.207057] ACPI: PM: (supports S0 S3 S4 S5) [ 0.208013] ACPI: Using IOAPIC for interrupt routing [ 0.210115] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.214454] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.224476] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.227043] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.229018] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.232078] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.237048] acpiphp: Slot [2] registered [ 0.238099] acpiphp: Slot [5] registered [ 0.240090] acpiphp: Slot [6] registered [ 0.241114] acpiphp: Slot [7] registered [ 0.243119] acpiphp: Slot [8] registered [ 0.244108] acpiphp: Slot [9] registered [ 0.245108] acpiphp: Slot [10] registered [ 0.247105] acpiphp: Slot [3] registered [ 0.248106] acpiphp: Slot [4] registered [ 0.249078] acpiphp: Slot [11] registered [ 0.251122] acpiphp: Slot [12] registered [ 0.252091] acpiphp: Slot [13] registered [ 0.254099] acpiphp: Slot [14] registered [ 0.255116] acpiphp: Slot [15] registered [ 0.256074] acpiphp: Slot [16] registered [ 0.257176] acpiphp: Slot [17] registered [ 0.259107] acpiphp: Slot [18] registered [ 0.260093] acpiphp: Slot [19] registered [ 0.262095] acpiphp: Slot [20] registered [ 0.264102] acpiphp: Slot [21] registered [ 0.266115] acpiphp: Slot [22] registered [ 0.267133] acpiphp: Slot [23] registered [ 0.269176] acpiphp: Slot [24] registered [ 0.271117] acpiphp: Slot [25] registered [ 0.272098] acpiphp: Slot [26] registered [ 0.274111] acpiphp: Slot [27] registered [ 0.275094] acpiphp: Slot [28] registered [ 0.277116] acpiphp: Slot [29] registered [ 0.279103] acpiphp: Slot [30] registered [ 0.280109] acpiphp: Slot [31] registered [ 0.282059] PCI host bridge to bus 0000:00 [ 0.283019] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.286032] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.289030] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.291042] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.293019] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.294018] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.296184] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.299024] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.302384] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.311015] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.316011] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.318018] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.321023] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.323017] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.326601] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.330044] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.332044] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.334802] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.339016] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.348873] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.353017] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.359195] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.384019] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.407019] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.437018] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.447899] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.455024] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.462018] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.490023] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.502000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.508015] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.513018] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.529021] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.539630] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.548031] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.556020] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.585022] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.594905] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.601019] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.607016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.627027] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.637334] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.645017] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.651019] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.666016] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.677941] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.679348] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.681343] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.683394] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.685223] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.690132] iommu: Default domain type: Passthrough [ 0.693566] SCSI subsystem initialized [ 0.695145] ACPI: bus type USB registered [ 0.696129] usbcore: registered new interface driver usbfs [ 0.698088] usbcore: registered new interface driver hub [ 0.700136] usbcore: registered new device driver usb [ 0.702185] pps_core: LinuxPPS API ver. 1 registered [ 0.704012] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.707067] PTP clock support registered [ 0.710071] EDAC MC: Ver: 3.0.0 [ 0.712131] PCI: Using ACPI for IRQ routing [ 0.713821] NetLabel: Initializing [ 0.715011] NetLabel: domain hash size = 128 [ 0.716010] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.718081] NetLabel: unlabeled traffic allowed by default [ 0.720140] vgaarb: loaded [ 0.722190] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.724009] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.731345] clocksource: Switched to clocksource kvm-clock [ 0.841712] VFS: Disk quotas dquot_6.6.0 [ 0.843195] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.845433] *** VALIDATE ramfs *** [ 0.846455] *** VALIDATE hugetlbfs *** [ 0.847647] pnp: PnP ACPI init [ 0.850088] pnp: PnP ACPI: found 6 devices [ 0.872327] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.875819] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.878079] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.879793] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.881883] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.883856] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.886538] NET: Registered protocol family 2 [ 0.888819] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.892480] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.895222] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.900622] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.904203] TCP: Hash tables configured (established 65536 bind 65536) [ 0.907243] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.910586] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.913447] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.916402] NET: Registered protocol family 1 [ 0.918936] RPC: Registered named UNIX socket transport module. [ 0.921198] RPC: Registered udp transport module. [ 0.922981] RPC: Registered tcp transport module. [ 0.924535] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.926099] NET: Registered protocol family 44 [ 0.927873] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.930029] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.932276] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.934594] PCI: CLS 0 bytes, default 64 [ 0.936976] Unpacking initramfs... [ 2.269549] debug: unmapping init [mem 0xffff8cc7fcc54000-0xffff8cc7fffbffff] [ 2.273862] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.276282] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.279390] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.763221] Initialise system trusted keyrings [ 2.765135] Key type blacklist registered [ 2.767316] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.775951] zbud: loaded [ 2.779042] *** VALIDATE nfs *** [ 2.780288] *** VALIDATE nfs4 *** [ 2.781854] pstore: using deflate compression [ 2.785933] Platform Keyring initialized [ 2.890050] NET: Registered protocol family 38 [ 2.892195] Key type asymmetric registered [ 2.893737] Asymmetric key parser 'x509' registered [ 2.895687] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.899800] io scheduler mq-deadline registered [ 2.901609] io scheduler kyber registered [ 2.903354] io scheduler bfq registered [ 2.904797] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.907698] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.910612] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.913655] ACPI: Power Button [PWRF] [ 2.918233] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.923893] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.932963] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.937595] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.948143] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.973798] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.999982] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.005803] Non-volatile memory driver v1.3 [ 3.006864] Linux agpgart interface v0.103 [ 3.040278] virtio_blk virtio1: [vda] 146936 512-byte logical blocks (75.2 MB/71.7 MiB) [ 3.042911] vda: detected capacity change from 0 to 75231232 [ 3.060339] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.063149] vdb: detected capacity change from 0 to 1073741824 [ 3.078229] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.081937] vdc: detected capacity change from 0 to 2621440000 [ 3.095200] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.098023] vdd: detected capacity change from 0 to 2621440000 [ 3.112553] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.115420] vde: detected capacity change from 0 to 4294967296 [ 3.128908] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.131854] vdf: detected capacity change from 0 to 4294967296 [ 3.138227] libphy: Fixed MDIO Bus: probed [ 3.143216] usbcore: registered new interface driver usbserial_generic [ 3.145119] usbserial: USB Serial support registered for generic [ 3.147202] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.150578] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.152039] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.154089] mousedev: PS/2 mouse device common for all mice [ 3.157206] rtc_cmos 00:05: RTC can wake from S4 [ 3.160655] rtc_cmos 00:05: registered as rtc0 [ 3.162702] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.163093] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.168450] intel_pstate: CPU model not supported [ 3.171267] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.173665] hid: raw HID events driver (C) Jiri Kosina [ 3.176408] usbcore: registered new interface driver usbhid [ 3.176474] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.178450] usbhid: USB HID core driver [ 3.182874] drop_monitor: Initializing network drop monitor service [ 3.185273] Initializing XFRM netlink socket [ 3.187123] NET: Registered protocol family 10 [ 3.189672] Segment Routing with IPv6 [ 3.191238] NET: Registered protocol family 17 [ 3.193227] mpls_gso: MPLS GSO support [ 3.197963] RAS: Correctable Errors collector initialized. [ 3.200489] AVX version of gcm_enc/dec engaged. [ 3.202326] AES CTR mode by8 optimization enabled [ 3.272790] sched_clock: Marking stable (3272759325, 0)->(4159562131, -886802806) [ 3.277280] registered taskstats version 1 [ 3.280154] Loading compiled-in X.509 certificates [ 3.282462] zswap: loaded using pool lzo/zbud [ 3.304808] Key type big_key registered [ 3.317099] Key type encrypted registered [ 3.318832] ima: No TPM chip found, activating TPM-bypass! [ 3.321066] ima: Allocated hash algorithm: sha1 [ 3.322847] ima: No architecture policies found [ 3.324697] evm: Initialising EVM extended attributes: [ 3.326718] evm: security.selinux [ 3.327916] evm: security.ima [ 3.329058] evm: security.capability [ 3.330332] evm: HMAC attrs: 0x1 [ 3.332917] rtc_cmos 00:05: setting system clock to 2026-08-27 15:22:31 UTC (1787844151) [ 3.338497] debug: unmapping init [mem 0xffffffff95a03000-0xffffffff95bfffff] [ 3.341709] debug: unmapping init [mem 0xffffffff94782000-0xffffffff94a58fff] [ 3.354121] Write protecting the kernel read-only data: 28672k [ 3.358178] debug: unmapping init [mem 0xffffffff92e03000-0xffffffff92ffffff] [ 3.361210] debug: unmapping init [mem 0xffffffff93714000-0xffffffff937fffff] [ 3.395726] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.403758] systemd[1]: Detected virtualization kvm. [ 3.405765] systemd[1]: Detected architecture x86-64. [ 3.407733] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.433694] systemd[1]: No hostname configured. [ 3.435276] systemd[1]: Set hostname to . [ 3.436436] random: systemd: uninitialized urandom read (16 bytes read) [ 3.437797] systemd[1]: Initializing machine ID from random generator. [ 3.471217] random: ln: uninitialized urandom read (6 bytes read) [ 3.557122] random: systemd: uninitialized urandom read (16 bytes read) [ 3.560525] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.564480] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 3.568800] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Swap. [ OK ] Reached target Timers. [ OK ] Reached target Initrd Root Device. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket. Starting Journal Service... Starting Setup Virtual Console... Starting Create Volatile Files and Directories... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.044310] device-mapper: uevent: version 1.0.3 [ 4.046428] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 4.707965] virtio_net virtio0 ens2: renamed from eth0 [ 4.722918] random: fast init done [ 4.765322] scsi host0: ata_piix [ 4.771847] scsi host1: ata_piix [ 4.773347] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 4.775997] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.058059] dracut-initqueue[591]: RTNETLINK answers: File exists [ 9.839256] random: crng init done [ 9.839998] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.080616] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Slices. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.195894] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.414583] SELinux: Disabled at runtime. [ 11.470447] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.477879] systemd[1]: Detected virtualization kvm. [ 11.478944] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 11.866981] systemd[1]: initrd-switch-root.service: Succeeded. [ 11.869934] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 11.876646] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 11.880230] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 11.883038] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 11.897137] systemd[1]: Starting Journal Service... Starting Journal Service... [ 11.905526] systemd[1]: Starting Create list of required static device nodes for the current kernel... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on udev Kernel Socket. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Activating swap /dev/disk/by-label/SWAP... Mounting POSIX Message Queue File System... Starting Remount Root and Kernel File Systems... [ OK ] Listening on udev Control Socket. [ 11.990724] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting udev Coldplug all Devices... [ OK ] Created slice system-getty.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on initctl Compatibility Named Pipe. Mounting Huge Pages File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. Starting Apply Kernel Variables... Mounting Kernel Debug File System... [ OK ] Reached target RPC Port Mapper. [ OK ] Reached target Paths. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Kernel Debug File System. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt.[ 12.276728] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 12.630146] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 12.655623] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 12.729403] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 12.746971] EDAC sbridge: Ver: 1.1.2 [ 13.967778] Key type dns_resolver registered [ 14.277972] NFS: Registering the id_resolver key type [ 14.280263] Key type id_resolver registered [ 14.282092] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. [ OK ] Started irqbalance daemon. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg257-server login: [ 41.712853] libcfs: loading out-of-tree module taints kernel. [ 41.729600] Key type ._llcrypt registered [ 41.730984] Key type .llcrypt registered [ 41.775275] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_hostid [ 61.127504] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 62.912500] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 62.965720] alg: No test for adler32 (adler32-zlib) [ 64.507204] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [ 65.392097] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [ 67.207926] Key type lgssc registered [ 69.383143] Lustre: Echo OBD driver; http://www.lustre.org/ [ 88.499355] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 132.848515] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 147.627834] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 147.710852] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 149.095640] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 149.139141] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 149.328477] Lustre: lustre-MDT0000: new disk, initializing [ 149.498235] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 149.526283] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 155.626236] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 155.892597] hrtimer: interrupt took 6466033 ns [ 173.405064] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 173.607630] Lustre: 6516:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 173.651808] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 173.655759] Lustre: Skipped 1 previous similar message [ 173.742463] Lustre: lustre-MDT0001: new disk, initializing [ 173.869659] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 173.911229] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 173.935758] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 178.222552] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 182.661170] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 195.755191] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 196.066206] Lustre: lustre-OST0000: new disk, initializing [ 196.077586] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 196.089285] Lustre: 8453:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 196.128040] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 203.404122] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 203.436560] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 203.653384] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 203.858632] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 225.092455] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 225.284925] Lustre: lustre-OST0001: new disk, initializing [ 225.295787] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 225.305564] Lustre: 9527:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 225.443280] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 233.533533] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 234.024211] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 234.027879] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 234.092699] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 248.408994] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 256.034827] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 264.978816] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing check_logdir /tmp/testlogs/ [ 271.770521] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing yml_node [ 276.996150] Lustre: DEBUG MARKER: Client: 2.17.57.89 [ 280.648763] Lustre: DEBUG MARKER: MDS: 2.17.57.89 [ 284.446269] Lustre: DEBUG MARKER: OSS: 2.17.57.89 [ 286.514905] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Aug 27 11:27:12 EDT 2026 [ 307.995858] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 309.942599] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 327.662038] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 327.669065] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 327.682566] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 331.232678] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 331.235482] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 331.254528] Lustre: Skipped 1 previous similar message [ 331.273892] Lustre: Skipped 3 previous similar messages [ 332.763872] Lustre: server umount lustre-MDT0000 complete [ 337.503217] LustreError: 6508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787844485 with bad export cookie 8219754596258014768 [ 337.513042] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 337.521804] LustreError: 6508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 341.473489] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 341.477217] LustreError: 7940:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 341.504716] Lustre: Skipped 1 previous similar message [ 341.508059] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 341.539928] LustreError: 7940:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 6 previous similar messages [ 342.500448] LustreError: 6521:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 342.517747] LustreError: 6521:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [ 343.979168] Lustre: server umount lustre-MDT0001 complete [ 348.367428] Lustre: server umount lustre-OST0000 complete [ 352.212754] Lustre: server umount lustre-OST0001 complete [ 359.527298] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_hostid [ 369.963956] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 410.937754] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 411.187963] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 411.220516] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 411.369463] Lustre: lustre-MDT0000: new disk, initializing [ 411.569318] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 411.596287] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 417.516274] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 429.529464] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 429.654685] Lustre: 15084:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 429.722816] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 429.725636] Lustre: Skipped 1 previous similar message [ 429.828508] Lustre: lustre-MDT0001: new disk, initializing [ 429.982643] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 430.050074] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 430.066118] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 434.794909] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 444.801761] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 450.713501] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 458.765920] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 459.188442] Lustre: lustre-OST0000: new disk, initializing [ 459.191327] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 459.197892] Lustre: 17053:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 459.339876] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 460.970878] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 460.986205] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 461.147753] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 466.482694] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 477.312987] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 483.157688] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 483.369674] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 488.434971] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 488.678176] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 492.004922] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 492.009452] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 492.009460] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 492.009467] Lustre: Skipped 3 previous similar messages [ 492.018438] Lustre: Skipped 2 previous similar messages [ 492.048369] LustreError: Skipped 1 previous similar message [ 497.123599] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 497.142169] Lustre: Skipped 2 previous similar messages [ 505.311177] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 505.724494] Lustre: server umount lustre-OST0000 complete [ 511.968595] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 511.978153] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 511.994975] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 512.003174] Lustre: Skipped 2 previous similar messages [ 524.255200] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 524.491137] Lustre: server umount lustre-MDT0000 complete [ 527.849197] LustreError: 15095:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 527.864417] LustreError: 15095:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [ 527.953485] LustreError: 15075:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787844676 with bad export cookie 8219754596258016294 [ 527.962910] LustreError: 15075:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 527.965329] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 528.331638] Lustre: server umount lustre-MDT0001 complete [ 534.615252] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 11:31:21 (1787844681) [ 542.068474] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 542.507095] LustreError: 19321:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 542.582451] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 547.179759] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 547.808818] LustreError: 19322:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 557.028594] LustreError: 19321:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 557.042453] LustreError: 19321:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [ 557.239293] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 562.812603] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 571.705851] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 578.810021] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 587.399519] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 595.294889] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 604.005289] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 619.160803] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 627.684605] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 627.690306] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 627.710641] Lustre: Skipped 3 previous similar messages [ 627.750880] Lustre: Skipped 8 previous similar messages [ 633.226397] Lustre: server umount lustre-MDT0000 complete [ 637.498521] LustreError: 19301:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787844785 with bad export cookie 8219754596258017183 [ 637.508503] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 637.879713] Lustre: server umount lustre-MDT0001 complete [ 652.281594] Lustre: server umount lustre-OST0000 complete [ 675.112658] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 685.994302] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 686.331255] LustreError: 23726:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 686.385971] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 686.399712] Lustre: Skipped 2 previous similar messages [ 691.683576] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 700.729953] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 705.948466] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 709.664421] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 717.821316] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 725.973476] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 736.643774] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 736.849190] Lustre: lustre-OST0001: new disk, initializing [ 736.852832] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 736.859474] Lustre: 26047:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 738.338554] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 738.357026] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 738.480587] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 743.929240] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 754.030802] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 757.619093] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 773.088345] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 773.102409] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 773.128608] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 773.140357] Lustre: Skipped 3 previous similar messages [ 776.573378] Lustre: server umount lustre-MDT0000 complete [ 779.244561] LustreError: 25466:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 779.254758] LustreError: 25466:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 4 previous similar messages [ 780.108924] LustreError: 24867:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787844928 with bad export cookie 8219754596258018030 [ 780.118789] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 780.123958] LustreError: 24867:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 780.445241] Lustre: server umount lustre-MDT0001 complete [ 794.018188] Lustre: server umount lustre-OST0000 complete [ 808.824241] Lustre: server umount lustre-OST0001 complete [ 826.891788] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 838.640921] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 839.138581] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 839.147281] Lustre: Skipped 3 previous similar messages [ 844.273390] LustreError: 29065:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 844.291909] LustreError: 29065:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [ 844.898295] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 855.004941] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 860.701583] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 864.607836] Lustre: 30213:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 872.587273] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 880.942504] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 890.639664] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 898.248722] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 907.053829] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 910.818958] Lustre: 32094:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 922.083932] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 922.087636] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 922.101721] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 922.101731] Lustre: Skipped 4 previous similar messages [ 922.109906] Lustre: Skipped 4 previous similar messages [ 925.113175] Lustre: server umount lustre-MDT0000 complete [ 929.863956] LustreError: 32096:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787845078 with bad export cookie 8219754596258019325 [ 929.878605] LustreError: 32096:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 929.886455] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 930.526936] Lustre: server umount lustre-MDT0001 complete [ 944.977263] Lustre: server umount lustre-OST0000 complete [ 948.642111] Lustre: 3649:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787845080/real 1787845080] req@ffff8cc84d8c70c0 x1874690533810688/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787845096 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 949.857186] Lustre: server umount lustre-OST0001 complete [ 962.616411] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 11:38:28 (1787845108) [ 1000.978161] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 1015.478217] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1016.103978] LustreError: 34902:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1016.120736] LustreError: 34902:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 11 previous similar messages [ 1021.310591] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 1030.718927] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1035.248338] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 1038.556451] Lustre: 36051:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1046.022650] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1054.086333] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 1066.233552] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1075.212405] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 1083.772786] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1088.098240] Lustre: 37931:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1097.723472] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1097.732587] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1097.752019] Lustre: Skipped 5 previous similar messages [ 1097.757817] Lustre: Skipped 4 previous similar messages [ 1097.780509] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1101.251651] Lustre: server umount lustre-MDT0000 complete [ 1104.928027] LustreError: 37934:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787845253 with bad export cookie 8219754596258020431 [ 1104.929528] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1104.939473] LustreError: 37934:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1105.305927] Lustre: server umount lustre-MDT0001 complete [ 1119.445578] Lustre: server umount lustre-OST0000 complete [ 1133.930235] Lustre: server umount lustre-OST0001 complete [ 1144.786529] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 11:41:30 (1787845290) [ 1163.654464] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 1175.646501] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1176.080463] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1176.083510] Lustre: Skipped 7 previous similar messages [ 1181.358880] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 1190.953888] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1196.049849] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 1198.810703] Lustre: 41225:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1205.938975] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1213.192479] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 1221.217993] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1228.056157] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 1235.577955] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1239.491088] Lustre: 43103:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1240.496519] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1240.508143] Lustre: Skipped 1 previous similar message [ 1247.713293] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1247.729466] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1247.754240] Lustre: Skipped 1 previous similar message [ 1250.032970] Lustre: server umount lustre-MDT0000 complete [ 1254.071661] LustreError: 40058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787845402 with bad export cookie 8219754596258021537 [ 1254.085398] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1260.468043] Lustre: server umount lustre-MDT0001 complete [ 1270.349761] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1278.943788] LustreError: 3645:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8cc87fc1a580 x1874690533913600/t0(0) o250->MGC192.168.202.157@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1279.398438] LustreError: 41588:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1279.414808] LustreError: 41588:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 26 previous similar messages [ 1284.718549] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 1289.723452] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1293.310447] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1298.932884] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1298.940155] Lustre: Skipped 1 previous similar message [ 1298.993660] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 1308.194891] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1309.922166] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1316.094654] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1318.538620] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1324.515959] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1330.368978] Lustre: server umount lustre-MDT0000 complete [ 1335.952470] LustreError: 40058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787845484 with bad export cookie 8219754596258022636 [ 1335.956702] LustreError: 40058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 1335.961092] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1336.566293] Lustre: server umount lustre-MDT0001 complete [ 1349.209955] Lustre: server umount lustre-OST0000 complete [ 1364.047946] Lustre: server umount lustre-OST0001 complete [ 1373.962816] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 11:45:20 (1787845520) [ 1392.053228] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 1403.151600] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1408.945593] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 1418.763952] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1424.424535] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 1427.770759] Lustre: 48864:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1435.317803] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1442.722831] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 1453.086971] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1460.436651] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 1469.358764] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1474.560055] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1489.384098] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1489.392964] Lustre: Skipped 21 previous similar messages [ 1495.612102] Lustre: server umount lustre-MDT0000 complete [ 1499.952365] LustreError: 47692:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787845648 with bad export cookie 8219754596258023462 [ 1499.968151] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1499.974535] LustreError: 47692:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1500.377218] Lustre: server umount lustre-MDT0001 complete [ 1514.166877] Lustre: server umount lustre-OST0000 complete [ 1528.254483] Lustre: server umount lustre-OST0001 complete [ 1540.529414] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 11:48:06 (1787845686) [ 1543.397755] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1545.958544] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 11:48:12 (1787845692) [ 1558.859972] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_hostid [ 1571.858829] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 1600.327353] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1600.728658] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1600.825946] Lustre: lustre-MDT0000: new disk, initializing [ 1600.949277] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1606.580994] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 1617.539362] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1626.286858] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1626.689446] Lustre: lustre-OST0000: new disk, initializing [ 1626.701677] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1626.704133] Lustre: Skipped 1 previous similar message [ 1626.711267] Lustre: 54949:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1628.572837] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1628.578214] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1628.609798] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1634.360661] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 1645.751765] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1649.388445] Lustre: DEBUG MARKER: create test files [ 1690.081853] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1690.082349] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1690.100923] Lustre: Skipped 13 previous similar messages [ 1696.463873] Lustre: server umount lustre-OST0000 complete [ 1701.324439] Lustre: server umount lustre-MDT0000 complete [ 1715.357571] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [ 1718.921056] Key type lgssc unregistered [ 1719.325798] LNet: 56585:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1719.331829] LNetError: 56585:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1720.360230] LNet: Removed LNI 192.168.202.157@tcp [ 1721.774150] Key type .llcrypt unregistered [ 1721.775974] Key type ._llcrypt unregistered [ 1757.050558] Key type ._llcrypt registered [ 1757.054078] Key type .llcrypt registered [ 1757.178716] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 1758.357736] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1758.512752] alg: No test for adler32 (adler32-zlib) [ 1759.700538] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [ 1760.129198] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [ 1761.855709] Key type lgssc registered [ 1763.039710] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1771.770958] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1771.823713] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1773.513364] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1778.288963] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 1787.018832] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1795.744176] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1796.259731] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1798.340631] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1798.494265] Lustre: 59319:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1798.503413] Lustre: 59319:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1798.510503] Lustre: 59319:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1798.521029] Lustre: 59319:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1798.529680] Lustre: 59319:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1798.546432] Lustre: 59319:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1803.926934] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 1813.124946] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1819.861141] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1826.659980] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1839.587634] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1839.622842] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1840.980818] Lustre: server umount lustre-OST0000 complete [ 1846.145557] Lustre: server umount lustre-MDT0000 complete [ 1859.405327] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [ 1863.052494] Key type lgssc unregistered [ 1863.361738] LNet: 60933:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1863.370924] LNetError: 60933:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1863.402101] LNet: Removed LNI 192.168.202.157@tcp [ 1864.666355] Key type .llcrypt unregistered [ 1864.669644] Key type ._llcrypt unregistered [ 1899.344687] Key type ._llcrypt registered [ 1899.349988] Key type .llcrypt registered [ 1899.540515] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 1900.928426] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1901.127154] alg: No test for adler32 (adler32-zlib) [ 1902.176922] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [ 1902.473384] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [ 1904.160620] Key type lgssc registered [ 1904.885075] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1913.919287] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1913.939257] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1915.594978] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1921.326197] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 1929.793863] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1938.346348] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1938.769161] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1940.873323] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1940.966051] Lustre: 63768:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1940.975958] Lustre: 63768:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1940.981026] Lustre: 63768:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1940.989983] Lustre: 63768:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1940.997138] Lustre: 63768:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1941.003084] Lustre: 63768:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1946.069067] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 1954.892891] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1961.091925] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1976.808380] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1976.822598] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1976.852795] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1978.337475] Lustre: server umount lustre-OST0000 complete [ 1982.636289] Lustre: server umount lustre-MDT0000 complete [ 1994.907741] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [ 1998.164216] Key type lgssc unregistered [ 1998.480842] LNet: 65333:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1998.487967] LNetError: 65333:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1998.510360] LNet: Removed LNI 192.168.202.157@tcp [ 1999.581385] Key type .llcrypt unregistered [ 1999.582869] Key type ._llcrypt unregistered [ 2021.142571] Key type ._llcrypt registered [ 2021.150835] Key type .llcrypt registered [ 2021.349929] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_hostid [ 2039.513657] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 2041.153278] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2041.197087] alg: No test for adler32 (adler32-zlib) [ 2042.450679] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [ 2042.731763] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [ 2044.463322] Key type lgssc registered [ 2045.526640] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2085.698708] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2085.740683] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2087.077294] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2087.101313] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2087.173235] Lustre: lustre-MDT0000: new disk, initializing [ 2087.231686] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2087.251775] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2092.550658] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 2105.248532] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2105.593761] Lustre: 68487:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2105.836474] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2105.844473] Lustre: Skipped 1 previous similar message [ 2106.005593] Lustre: lustre-MDT0001: new disk, initializing [ 2106.263074] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2106.322207] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2106.347238] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2112.849846] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 2124.927444] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2132.479760] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2140.717211] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2141.083381] Lustre: lustre-OST0000: new disk, initializing [ 2141.088484] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2141.100530] Lustre: 70458:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2141.189735] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2147.454269] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2147.463110] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2147.618470] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2148.558844] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 2160.050856] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2166.884602] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2167.131721] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2173.991709] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2174.266422] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2178.531267] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2178.532553] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2178.541780] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2178.551640] Lustre: Skipped 1 previous similar message [ 2178.564688] Lustre: Skipped 1 previous similar message [ 2183.652214] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2183.663982] Lustre: Skipped 1 previous similar message [ 2188.768985] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2188.785994] Lustre: Skipped 1 previous similar message [ 2191.328532] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2191.605581] Lustre: server umount lustre-OST0000 complete [ 2198.496494] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2198.503414] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2198.529170] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2204.132095] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2204.141335] Lustre: Skipped 3 previous similar messages [ 2211.298987] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2211.757090] Lustre: server umount lustre-MDT0000 complete [ 2216.153502] LustreError: 68480:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787846364 with bad export cookie 1026413485654300695 [ 2216.155251] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2216.170071] LustreError: 68480:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2216.593306] Lustre: server umount lustre-MDT0001 complete [ 2226.374308] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 11:59:32 (1787846372) [ 2239.758930] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 2289.439464] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_hostid [ 2301.399627] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 2344.178787] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2344.493734] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2344.574525] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2344.670420] Lustre: lustre-MDT0000: new disk, initializing [ 2344.757584] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2344.769434] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2350.297466] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 2363.294640] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2363.409677] Lustre: 76495:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2363.461914] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2363.465587] Lustre: Skipped 1 previous similar message [ 2363.564435] Lustre: lustre-MDT0001: new disk, initializing [ 2363.665335] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2363.764180] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2363.786900] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2369.204047] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 2380.146609] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2386.041123] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2394.121749] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2394.457221] Lustre: lustre-OST0000: new disk, initializing [ 2394.461537] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2394.467422] Lustre: 78467:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2394.555921] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2400.685632] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 2403.978596] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2403.992735] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2404.197939] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2411.404687] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2416.872258] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2417.134606] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2422.871428] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2423.126572] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2429.924897] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2429.931370] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2429.950535] LustreError: Skipped 1 previous similar message [ 2429.951521] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2429.951526] Lustre: Skipped 2 previous similar messages [ 2430.005798] Lustre: Skipped 2 previous similar messages [ 2440.161638] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2440.495323] Lustre: server umount lustre-OST0000 complete [ 2445.281967] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2450.401798] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2450.406179] Lustre: Skipped 8 previous similar messages [ 2451.812300] Lustre: server umount lustre-MDT0000 complete [ 2455.538545] LustreError: 76508:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2455.559537] LustreError: 76508:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 4 previous similar messages [ 2456.434046] LustreError: 76488:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787846604 with bad export cookie 1026413485654301570 [ 2456.444114] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2456.451023] LustreError: 76488:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2456.943344] Lustre: server umount lustre-MDT0001 complete [ 2467.732885] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 12:03:34 (1787846614) [ 2475.488614] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2475.990417] LustreError: 80882:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2476.076422] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2481.123421] LustreError: 80883:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2481.147174] LustreError: 80883:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [ 2481.567433] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 2486.243097] LustreError: 80882:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2491.015495] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2496.642685] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 2505.209339] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2511.194766] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2519.115229] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2519.440244] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2519.451982] Lustre: Skipped 1 previous similar message [ 2526.578443] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 2535.434562] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2539.597257] Lustre: 83554:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting e34c5424-977e-4bdf-99e3-d8535a71ecfe at adminstrative request [ 2541.055714] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2541.077665] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2551.269900] Lustre: MGS: Client e34c5424-977e-4bdf-99e3-d8535a71ecfe (at 0@lo) reconnecting [ 2551.330100] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xe3e8da085185ef4 to 0xe3e8da08518619b [ 2551.342782] Lustre: MGC192.168.202.157@tcp: Connection restored to 0@lo (at 0@lo) [ 2579.236456] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2579.492428] Lustre: lustre-OST0001: new disk, initializing [ 2579.495482] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2579.502910] Lustre: 83952:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2586.528486] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 2586.707864] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2586.721135] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2586.847916] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2597.084084] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2602.469982] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2602.470152] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2602.477057] Lustre: Skipped 1 previous similar message [ 2602.481847] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2602.490040] LustreError: Skipped 1 previous similar message [ 2602.492532] Lustre: Skipped 1 previous similar message [ 2606.483692] Lustre: server umount lustre-OST0001 complete [ 2607.589075] LustreError: 82703:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2607.603588] LustreError: 82703:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [ 2612.712811] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2612.722119] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2612.742133] Lustre: Skipped 2 previous similar messages [ 2616.456218] Lustre: server umount lustre-OST0000 complete [ 2628.065053] LustreError: 80882:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2628.084511] LustreError: 80882:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 2 previous similar messages [ 2628.128519] Lustre: server umount lustre-MDT0000 complete [ 2632.798087] LustreError: 80864:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787846780 with bad export cookie 1026413485654303131 [ 2632.805157] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2632.809570] LustreError: 80864:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2633.202480] Lustre: server umount lustre-MDT0001 complete [ 2642.352477] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 12:06:28 (1787846788) [ 2644.431259] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2646.813323] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 12:06:33 (1787846793) [ 2648.870100] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2651.248380] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 12:06:37 (1787846797) [ 2653.474528] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2655.508194] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 12:06:41 (1787846801) [ 2658.014622] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2660.617558] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 12:06:46 (1787846806) [ 2662.835512] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2672.248277] loop: module loaded [ 2672.279732] loop0: detected capacity change from 0 to 4294967296 [ 2672.352665] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2673.978895] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2678.186904] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2684.134471] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2716.870547] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2716.957075] LustreError: 88077:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2716.967799] LustreError: 88077:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2716.972623] LustreError: 88077:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2716.979390] LustreError: 88077:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2739.687500] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_hostid [ 2750.939349] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 2795.241885] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2795.623618] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2795.669041] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2795.843984] Lustre: lustre-MDT0000: new disk, initializing [ 2796.099822] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2796.112842] Lustre: Skipped 1 previous similar message [ 2796.141090] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2801.976850] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 2814.829914] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2814.899238] Lustre: 90945:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2814.928362] LustreError: 90956:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2814.952675] LustreError: 90956:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 3 previous similar messages [ 2815.090174] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2820.902045] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 2831.455442] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2839.752644] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2848.289406] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2848.599547] Lustre: 92916:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2856.294669] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 2859.105762] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2859.288772] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2867.405862] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2872.993510] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2873.319298] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2878.976150] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2879.215663] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2885.091124] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2885.104143] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2885.133022] Lustre: Skipped 2 previous similar messages [ 2885.144618] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2885.159543] Lustre: Skipped 5 previous similar messages [ 2888.055282] Lustre: server umount lustre-OST0000 complete [ 2900.452584] LustreError: 90957:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2900.477073] LustreError: 90957:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [ 2900.749540] Lustre: server umount lustre-MDT0000 complete [ 2904.359657] LustreError: 90937:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787847052 with bad export cookie 1026413485654303747 [ 2904.372787] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2904.375594] LustreError: 90937:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2904.707134] Lustre: server umount lustre-MDT0001 complete [ 2914.685458] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 12:11:01 (1787847061) [ 2921.693900] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2927.230992] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 2940.753224] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2941.290488] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2941.293406] Lustre: Skipped 3 previous similar messages [ 2947.383696] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 2955.327448] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2962.357763] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2970.917462] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2979.415844] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 2987.615905] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2993.876666] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2994.179474] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2999.564465] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2999.888779] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3007.880375] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3008.032759] Lustre: lustre-OST0001: new disk, initializing [ 3008.035146] Lustre: Skipped 2 previous similar messages [ 3008.038937] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3008.042629] Lustre: Skipped 3 previous similar messages [ 3008.047532] Lustre: 98704:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3014.343784] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3014.349453] Lustre: Skipped 2 previous similar messages [ 3014.359092] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3014.538865] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3016.282428] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 3027.507206] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3033.714975] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 3033.893929] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3040.097802] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 3040.488898] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3061.332604] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3087.082753] Lustre: Failing over lustre-MDT0000 [ 3087.486504] Lustre: server umount lustre-MDT0000 complete [ 3089.177331] LustreError: 97988:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.57@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3089.198291] LustreError: 97988:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 5 previous similar messages [ 3089.890159] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3089.897147] LustreError: Skipped 1 previous similar message [ 3089.901481] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3089.912639] Lustre: Skipped 3 previous similar messages [ 3107.807254] Lustre: 66713:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787847239/real 1787847239] req@00000000927b86db x1874692606819072/t0(0) o400->MGC192.168.202.157@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787847255 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3107.847333] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3114.790210] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3114.793706] LDISKFS-fs (dm-0): recovery complete [ 3114.813464] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3118.455692] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3120.886226] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3120.890638] Lustre: 95327:0:(ldlm_lib.c:2132:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3123.092930] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 3123.692954] Lustre: 96059:0:(ldlm_lib.c:2132:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3123.707496] Lustre: 96059:0:(ldlm_lib.c:2132:extend_recovery_timer()) Skipped 1 previous similar message [ 3123.739224] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3123.840882] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3124.160788] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3124.518839] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3124.537175] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 1 previous similar message [ 3124.872058] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3124.882649] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 1 previous similar message [ 3124.902799] Lustre: 101560:0:(ldlm_lib.c:2132:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3124.914167] Lustre: 101560:0:(ldlm_lib.c:2132:extend_recovery_timer()) Skipped 7 previous similar messages [ 3125.568156] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3125.575688] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 2 previous similar messages [ 3125.895352] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3125.903036] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 2 previous similar messages [ 3127.183162] Lustre: 101560:0:(ldlm_lib.c:2132:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3127.201094] Lustre: 101560:0:(ldlm_lib.c:2132:extend_recovery_timer()) Skipped 6 previous similar messages [ 3127.856942] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3127.876027] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 6 previous similar messages [ 3128.199953] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3128.215240] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 6 previous similar messages [ 3131.408522] Lustre: 101560:0:(ldlm_lib.c:2132:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3131.431068] Lustre: 101560:0:(ldlm_lib.c:2132:extend_recovery_timer()) Skipped 12 previous similar messages [ 3132.096793] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3132.112431] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 12 previous similar messages [ 3132.431138] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3132.444927] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 12 previous similar messages [ 3139.554780] Lustre: 101560:0:(ldlm_lib.c:2132:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3139.579470] Lustre: 101560:0:(ldlm_lib.c:2132:extend_recovery_timer()) Skipped 24 previous similar messages [ 3140.234143] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3140.259714] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 24 previous similar messages [ 3140.583466] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3140.596577] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 24 previous similar messages [ 3155.723081] Lustre: 101560:0:(ldlm_lib.c:2132:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3155.732757] Lustre: 101560:0:(ldlm_lib.c:2132:extend_recovery_timer()) Skipped 49 previous similar messages [ 3156.379703] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3156.394570] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 49 previous similar messages [ 3156.727225] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3156.738492] LustreError: 101560:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 49 previous similar messages [ 3180.335177] Lustre: 101560:0:(ldlm_lib.c:2441:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3180.346058] Lustre: 101560:0:(ldlm_lib.c:2451:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3180.368491] Lustre: 101560:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client d321c436-b5e8-4196-8ee3-51c5946b596a@192.168.202.57@tcp [ 3180.391565] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3180.408678] Lustre: 101560:0:(ldlm_lib.c:1961:abort_req_replay_queue()) @@@ aborted: req@00000000ef07cc01 x1874692588769408/t0(8589934772) o35->d321c436-b5e8-4196-8ee3-51c5946b596a@192.168.202.57@tcp:544/0 lens 392/0 e 0 to 0 dl 1787847339 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3180.460460] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3180.484451] Lustre: lustre-MDT0000: Denying connection for new client d321c436-b5e8-4196-8ee3-51c5946b596a (at 192.168.202.57@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3180.520190] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3180.568387] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3180.579104] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3180.591804] Lustre: Skipped 2 previous similar messages [ 3180.629966] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 3180.630878] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 3196.898308] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3196.908290] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3196.912893] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3196.919793] Lustre: Skipped 4 previous similar messages [ 3196.926217] Lustre: Skipped 2 previous similar messages [ 3210.210597] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3210.500293] Lustre: server umount lustre-OST0000 complete [ 3220.983599] LustreError: 97155:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3221.004622] LustreError: 97155:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 46 previous similar messages [ 3229.151313] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3229.431839] Lustre: server umount lustre-OST0001 complete [ 3242.124715] Lustre: server umount lustre-MDT0000 complete [ 3247.040384] LustreError: 98703:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787847395 with bad export cookie 1026413485654354966 [ 3247.054625] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3247.057862] LustreError: 98703:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3247.610406] Lustre: server umount lustre-MDT0001 complete [ 3258.687729] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 12:16:45 (1787847405) [ 3267.731505] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3267.745471] Lustre: *** cfs_fail_loc=197, val=0*** [ 3283.360823] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3288.480291] LustreError: 103537:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.157@tcp: failed processing log, type 4: rc = -110 [ 3293.617739] Lustre: *** cfs_fail_loc=197, val=0*** [ 3293.620583] Lustre: Skipped 65 previous similar messages [ 3314.207320] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3314.235468] Lustre: Skipped 3 previous similar messages [ 3321.051323] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 3329.736187] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3331.887710] Lustre: server umount lustre-OST0000 complete [ 3340.628222] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 12:18:07 (1787847487) [ 3362.267968] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3367.978929] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 3377.939149] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3382.054335] LustreError: 105187:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3382.066479] LustreError: 105187:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3382.070239] LustreError: 105187:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 3382.428855] Lustre: server umount lustre-MDT0000 complete [ 3416.242736] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_hostid [ 3427.406139] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 3471.421880] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3471.988606] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3472.093423] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3472.390616] Lustre: lustre-MDT0000: new disk, initializing [ 3472.707090] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3479.788401] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 3489.624865] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3489.715235] Lustre: 109365:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3489.935987] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3494.372356] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 3504.628849] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3511.644736] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3519.096604] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3519.389694] Lustre: 111332:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3526.068371] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 3527.951140] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3536.243675] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3541.529915] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3541.824355] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3548.674675] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3548.863089] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3553.769520] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3553.771255] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3553.771912] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3553.771918] Lustre: Skipped 13 previous similar messages [ 3553.789281] Lustre: Skipped 5 previous similar messages [ 3553.816950] LustreError: Skipped 1 previous similar message [ 3557.659633] Lustre: server umount lustre-OST0000 complete [ 3577.312721] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3577.505737] Lustre: server umount lustre-MDT0000 complete [ 3579.361199] LustreError: 109377:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3579.387760] LustreError: 109377:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 12 previous similar messages [ 3582.062884] LustreError: 109359:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787847730 with bad export cookie 1026413485654365578 [ 3582.066161] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3582.073570] LustreError: 109359:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3582.378730] Lustre: server umount lustre-MDT0001 complete [ 3592.678220] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 12:22:19 (1787847739) [ 3603.229123] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [ 3606.720711] Key type lgssc unregistered [ 3607.045772] LNet: 113825:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3607.058653] LNetError: 113825:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3607.078514] LNet: Removed LNI 192.168.202.157@tcp [ 3607.938829] Key type .llcrypt unregistered [ 3607.940788] Key type ._llcrypt unregistered [ 3626.291194] Key type ._llcrypt registered [ 3626.293082] Key type .llcrypt registered [ 3626.402696] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_hostid [ 3644.070522] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 3645.635595] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3645.687480] alg: No test for adler32 (adler32-zlib) [ 3646.721721] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [ 3647.012663] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [ 3648.703202] Key type lgssc registered [ 3650.528484] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3696.473417] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3696.486557] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3697.802544] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3697.825137] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3697.961356] Lustre: lustre-MDT0000: new disk, initializing [ 3698.074242] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3698.113202] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3703.690404] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 3717.374660] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3717.879172] Lustre: lustre-OST0000: new disk, initializing [ 3717.882800] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3717.885892] Lustre: Skipped 1 previous similar message [ 3717.891729] Lustre: 118002:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3717.983777] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3726.285589] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 3727.992291] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3728.014627] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3728.072299] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3738.778316] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3738.925203] Lustre: lustre-OST0001: new disk, initializing [ 3738.928108] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3738.932858] Lustre: 118864:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3739.003957] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3744.938558] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3744.949955] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3744.985378] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3746.629935] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 3760.608797] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3760.618905] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3760.627441] Lustre: Skipped 1 previous similar message [ 3760.645851] Lustre: Skipped 1 previous similar message [ 3763.989666] Lustre: server umount lustre-MDT0000 complete [ 3773.489674] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3779.998303] LustreError: 119474:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787847928 with bad export cookie 694950582694614982 [ 3780.003854] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3780.017323] LustreError: 119474:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3780.421459] Lustre: server umount lustre-OST0000 complete [ 3786.038494] Lustre: server umount lustre-OST0001 complete [ 3798.061919] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 12:25:44 (1787847944) [ 3824.419126] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 12:26:10 (1787847970) [ 3836.288442] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 3862.603171] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 12:26:48 (1787848008) [ 3909.569246] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_hostid [ 3922.876672] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 3974.232659] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3974.622458] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3974.664700] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3974.794707] Lustre: lustre-MDT0000: new disk, initializing [ 3974.919758] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3974.960761] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3980.939702] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 3993.964706] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3994.159850] Lustre: 126623:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3994.204842] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3994.208639] Lustre: Skipped 1 previous similar message [ 3994.292321] Lustre: lustre-MDT0001: new disk, initializing [ 3994.474799] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3994.534858] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3994.560265] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4000.673802] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 4013.235479] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4021.342648] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4030.165534] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4030.565406] Lustre: lustre-OST0000: new disk, initializing [ 4030.569260] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4030.576696] Lustre: 128595:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4030.698660] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4037.196718] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4037.212677] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4037.341578] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4038.170545] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 4049.791612] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4055.524482] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4055.838298] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4060.532586] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4060.794611] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4063.202844] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4063.206845] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4063.221537] LustreError: Skipped 1 previous similar message [ 4063.237100] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4063.254615] Lustre: Skipped 1 previous similar message [ 4063.276508] Lustre: Skipped 1 previous similar message [ 4068.340465] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4068.355124] Lustre: Skipped 1 previous similar message [ 4069.188589] Lustre: server umount lustre-OST0000 complete [ 4073.446803] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4073.449988] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4073.460409] Lustre: Skipped 1 previous similar message [ 4078.561406] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4078.567635] Lustre: Skipped 2 previous similar messages [ 4079.388514] Lustre: server umount lustre-MDT0000 complete [ 4083.386685] LustreError: 126617:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787848231 with bad export cookie 694950582694616214 [ 4083.388877] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4083.397613] LustreError: 126617:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4083.712979] Lustre: server umount lustre-MDT0001 complete [ 4093.740922] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 12:30:39 (1787848239) [ 4102.110531] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4102.846335] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4107.966216] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 4108.260847] LustreError: 131010:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4108.272222] LustreError: 131010:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 2 previous similar messages [ 4112.362422] LustreError: 131009:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4117.484348] LustreError: 131010:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4118.371288] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4124.239852] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 4134.159298] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4140.741570] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4149.127739] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4149.616343] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4149.626680] Lustre: Skipped 1 previous similar message [ 4156.846967] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 4166.653621] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4176.809986] Lustre: *** cfs_fail_loc=159, val=0*** [ 4193.089835] Lustre: lustre-MDT0000: Client 647cf550-c8fa-434b-896d-bc03689d8996 (at 192.168.202.57@tcp) reconnecting [ 4195.072369] Lustre: *** cfs_fail_loc=159, val=0*** [ 4195.074113] Lustre: Skipped 5 previous similar messages [ 4211.535724] Lustre: lustre-MDT0000: Client 647cf550-c8fa-434b-896d-bc03689d8996 (at 192.168.202.57@tcp) reconnecting [ 4218.346528] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4218.377819] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4223.868566] Lustre: server umount lustre-OST0000 complete [ 4228.581366] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4228.606978] Lustre: Skipped 2 previous similar messages [ 4238.816648] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4238.823404] Lustre: Skipped 7 previous similar messages [ 4241.376236] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4241.685919] Lustre: server umount lustre-MDT0000 complete [ 4243.937133] LustreError: 131004:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4243.951121] LustreError: 131004:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [ 4246.693145] LustreError: 130991:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787848394 with bad export cookie 694950582694617096 [ 4246.705509] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4246.722146] LustreError: 130991:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4247.133732] Lustre: server umount lustre-MDT0001 complete [ 4265.583830] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [ 4270.146392] Key type lgssc unregistered [ 4270.492326] LNet: 134813:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4270.505317] LNetError: 134813:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4271.534885] LNet: Removed LNI 192.168.202.157@tcp [ 4273.036644] Key type .llcrypt unregistered [ 4273.038175] Key type ._llcrypt unregistered [ 4305.554529] Key type ._llcrypt registered [ 4305.556611] Key type .llcrypt registered [ 4305.658668] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 4306.592875] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4306.622596] alg: No test for adler32 (adler32-zlib) [ 4307.591579] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [ 4307.801430] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [ 4309.575199] Key type lgssc registered [ 4310.926413] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4321.084901] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4321.128672] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4322.877777] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4329.199362] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 4332.017340] LustreError: 136233:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4332.041760] LustreError: 136233:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 2 previous similar messages [ 4337.124820] LustreError: 136234:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4340.836199] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4341.303869] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4347.333873] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 4357.592312] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4365.089082] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4376.955195] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4377.601726] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4385.218623] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 4392.954797] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4394.615310] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4405.263742] Lustre: *** cfs_fail_loc=159, val=0*** [ 4420.907772] Lustre: lustre-MDT0000: Client 44a801d5-27fb-4a2c-9ba1-256d39968f84 (at 192.168.202.57@tcp) reconnecting [ 4424.342764] Lustre: *** cfs_fail_loc=159, val=0*** [ 4439.861420] Lustre: lustre-MDT0001: Client 44a801d5-27fb-4a2c-9ba1-256d39968f84 (at 192.168.202.57@tcp) reconnecting [ 4442.137277] Lustre: *** cfs_fail_loc=159, val=0*** [ 4442.146213] Lustre: Skipped 3 previous similar messages [ 4458.842061] Lustre: lustre-MDT0001: Client 44a801d5-27fb-4a2c-9ba1-256d39968f84 (at 192.168.202.57@tcp) reconnecting [ 4464.488315] Lustre: 139270:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4468.144640] Lustre: *** cfs_fail_loc=159, val=0*** [ 4468.146376] Lustre: Skipped 4 previous similar messages [ 4483.913928] Lustre: lustre-MDT0000: Client 16b56a55-390a-4063-a859-5c95ba9a2bda (at 192.168.202.57@tcp) reconnecting [ 4486.299138] Lustre: *** cfs_fail_loc=159, val=0*** [ 4486.307947] Lustre: Skipped 14 previous similar messages [ 4502.858359] Lustre: lustre-MDT0000: Client 16b56a55-390a-4063-a859-5c95ba9a2bda (at 192.168.202.57@tcp) reconnecting [ 4504.760900] Lustre: 139565:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4525.032864] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4525.049782] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4525.075214] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4526.064507] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4526.072115] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4526.114096] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4531.175351] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4531.184973] Lustre: Skipped 2 previous similar messages [ 4536.289996] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4536.294912] Lustre: Skipped 1 previous similar message [ 4537.311155] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4537.584703] Lustre: server umount lustre-OST0000 complete [ 4546.016533] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4546.031482] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4546.063910] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4549.204729] Lustre: server umount lustre-MDT0000 complete [ 4551.648276] LustreError: 139449:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4551.667264] LustreError: 139449:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [ 4553.652856] LustreError: 136213:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787848701 with bad export cookie 5420376091175165246 [ 4553.657935] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4553.675733] LustreError: 136213:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4554.052992] Lustre: server umount lustre-MDT0001 complete [ 4566.496294] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [ 4569.888180] Key type lgssc unregistered [ 4570.324519] LNet: 140647:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4570.332685] LNetError: 140647:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4570.362693] LNet: Removed LNI 192.168.202.157@tcp [ 4571.430608] Key type .llcrypt unregistered [ 4571.432852] Key type ._llcrypt unregistered [ 4601.149780] Key type ._llcrypt registered [ 4601.158419] Key type .llcrypt registered [ 4601.323480] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 4603.195332] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4603.232278] alg: No test for adler32 (adler32-zlib) [ 4604.540062] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [ 4604.913254] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [ 4606.673918] Key type lgssc registered [ 4608.191785] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4619.179585] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4619.197512] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4620.931699] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4628.064564] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 4631.547666] LustreError: 142064:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4631.582429] LustreError: 142064:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 2 previous similar messages [ 4636.643193] LustreError: 142065:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4639.369076] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4640.182629] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4646.633809] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 4657.319928] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4665.743699] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4675.804057] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4676.324306] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4685.171628] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 4686.588428] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4691.721293] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4693.127580] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4700.838545] Lustre: 144654:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4704.359045] Lustre: 144702:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4705.914030] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4706.938573] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4711.301352] Lustre: 144848:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4712.228394] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4713.134218] Lustre: 144946:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4717.537763] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4717.541597] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4717.546511] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4717.551792] Lustre: Skipped 1 previous similar message [ 4717.556091] LustreError: Skipped 1 previous similar message [ 4722.674611] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4722.687771] Lustre: Skipped 2 previous similar messages [ 4727.779759] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4727.800273] Lustre: Skipped 1 previous similar message [ 4731.359536] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4731.823198] Lustre: server umount lustre-OST0000 complete [ 4737.504563] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4737.526970] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4737.546912] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4743.140612] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4743.151247] Lustre: Skipped 3 previous similar messages [ 4743.495782] Lustre: server umount lustre-MDT0000 complete [ 4748.134708] LustreError: 142045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787848896 with bad export cookie 13470723881408834227 [ 4748.138973] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4748.141471] LustreError: 142045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4748.629125] Lustre: server umount lustre-MDT0001 complete [ 4764.094758] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [ 4767.922204] Key type lgssc unregistered [ 4768.350979] LNet: 146012:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4768.356172] LNetError: 146012:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4769.387445] LNet: Removed LNI 192.168.202.157@tcp [ 4770.863945] Key type .llcrypt unregistered [ 4770.866847] Key type ._llcrypt unregistered [ 4797.874972] Key type ._llcrypt registered [ 4797.881419] Key type .llcrypt registered [ 4798.034669] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 4799.723540] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4799.954225] alg: No test for adler32 (adler32-zlib) [ 4801.143905] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [ 4801.421409] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [ 4803.151699] Key type lgssc registered [ 4804.700887] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4815.462871] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4815.481451] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4817.182213] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4823.068626] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 4827.620513] LustreError: 147432:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4827.645026] LustreError: 147432:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 2 previous similar messages [ 4832.767592] LustreError: 147431:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4833.753141] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4834.386683] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4840.717302] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 4851.561085] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4858.788861] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4867.466239] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4868.080315] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4876.355453] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 4879.345882] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4886.259420] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4887.621080] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4894.357358] Lustre: *** cfs_fail_loc=159, val=0*** [ 4909.894488] Lustre: lustre-MDT0000: Client 8fddf641-8da4-4c42-a208-3c0feb71fd02 (at 192.168.202.57@tcp) reconnecting [ 4930.535234] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4930.542060] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4930.558656] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4933.603504] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4933.605725] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4933.606628] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4933.659951] Lustre: Skipped 1 previous similar message [ 4935.394755] Lustre: server umount lustre-OST0000 complete [ 4941.795996] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4941.802743] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4941.815054] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4943.847283] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4943.849540] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4943.909628] Lustre: Skipped 1 previous similar message [ 4946.403947] Lustre: server umount lustre-MDT0000 complete [ 4948.960668] LustreError: 147432:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4948.989599] LustreError: 147432:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [ 4950.537718] LustreError: 150053:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787849098 with bad export cookie 6023491186868508158 [ 4950.542463] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4950.556040] LustreError: 150053:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4950.781290] Lustre: server umount lustre-MDT0001 complete [ 4964.694611] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [ 4968.886388] Key type lgssc unregistered [ 4969.334257] LNet: 151120:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4969.354971] LNetError: 151120:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4969.373194] LNet: Removed LNI 192.168.202.157@tcp [ 4971.036758] Key type .llcrypt unregistered [ 4971.040172] Key type ._llcrypt unregistered [ 5003.736876] Key type ._llcrypt registered [ 5003.739420] Key type .llcrypt registered [ 5003.847536] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 5005.487964] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5005.524724] alg: No test for adler32 (adler32-zlib) [ 5006.731884] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [ 5007.030081] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [ 5008.720856] Key type lgssc registered [ 5010.622728] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5021.607719] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5021.635857] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5023.376894] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5029.891956] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 5032.938234] LustreError: 152538:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5032.974793] LustreError: 152538:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 2 previous similar messages [ 5038.049785] LustreError: 152537:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5040.412119] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5040.872762] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5046.773921] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 5056.627480] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5064.443670] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5073.691285] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5074.319964] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5082.246889] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 5085.553264] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 5091.177300] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5093.783787] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 5117.740602] Lustre: lustre-MDT0001: evicting f82c8d29-f130-4630-9703-c65567b46e1a (at 192.168.202.57@tcp) by administrative request [ 5117.752181] Lustre: Skipped 1 previous similar message [ 5150.180466] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5150.196511] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5150.196607] Lustre: Skipped 1 previous similar message [ 5150.200924] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5150.208837] Lustre: Skipped 1 previous similar message [ 5155.298972] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5155.323447] Lustre: Skipped 1 previous similar message [ 5155.753956] Lustre: server umount lustre-OST0000 complete [ 5163.504129] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5163.518132] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5163.537230] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5165.544379] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5165.564346] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5165.567047] Lustre: Skipped 1 previous similar message [ 5170.662313] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5170.667886] Lustre: Skipped 1 previous similar message [ 5175.265812] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5175.489424] Lustre: server umount lustre-MDT0000 complete [ 5175.781848] LustreError: 152537:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5175.802839] LustreError: 152537:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [ 5179.895398] LustreError: 152538:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5179.924571] LustreError: 152538:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [ 5181.214972] LustreError: 152518:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787849329 with bad export cookie 9397419421267197829 [ 5181.230913] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5181.234309] LustreError: 152518:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5181.669127] Lustre: server umount lustre-MDT0001 complete [ 5195.043494] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [ 5199.377289] Key type lgssc unregistered [ 5199.738903] LNet: 156615:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5199.763162] LNetError: 156615:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5199.790628] LNet: Removed LNI 192.168.202.157@tcp [ 5201.253688] Key type .llcrypt unregistered [ 5201.255389] Key type ._llcrypt unregistered [ 5286.211744] Key type ._llcrypt registered [ 5286.214401] Key type .llcrypt registered [ 5286.385856] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 5287.585632] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5287.610737] alg: No test for adler32 (adler32-zlib) [ 5289.070443] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [ 5289.347391] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [ 5291.087388] Key type lgssc registered [ 5292.824666] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5304.870408] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5304.921899] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5306.684203] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5313.720036] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 5316.074715] LustreError: 159106:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5316.102540] LustreError: 159106:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 2 previous similar messages [ 5321.186283] LustreError: 159107:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5325.918759] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5326.464850] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5333.773089] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 5343.484259] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5353.133567] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5366.291630] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5366.837105] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5370.957338] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5376.079953] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5376.676613] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 5390.835439] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5406.704637] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5406.705511] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5406.729313] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5406.737104] LustreError: Skipped 1 previous similar message [ 5406.753472] Lustre: Skipped 1 previous similar message [ 5409.159044] Lustre: server umount lustre-OST0000 complete [ 5416.929752] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5416.937703] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5416.972695] Lustre: Skipped 2 previous similar messages [ 5420.788793] Lustre: server umount lustre-MDT0000 complete [ 5422.051368] LustreError: 159103:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5422.066569] LustreError: 159103:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [ 5425.866368] LustreError: 159088:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787849574 with bad export cookie 6001624467027804895 [ 5425.871306] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5425.884688] LustreError: 159088:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5426.274239] Lustre: server umount lustre-MDT0001 complete [ 5439.193250] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [ 5441.915951] Key type lgssc unregistered [ 5442.398457] LNet: 162657:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5442.404464] LNetError: 162657:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5443.430536] LNet: Removed LNI 192.168.202.157@tcp [ 5444.964731] Key type .llcrypt unregistered [ 5444.970870] Key type ._llcrypt unregistered [ 5472.538857] Key type ._llcrypt registered [ 5472.541590] Key type .llcrypt registered [ 5472.639761] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 5473.753292] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5473.925579] alg: No test for adler32 (adler32-zlib) [ 5475.115382] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [ 5475.470944] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [ 5477.255449] Key type lgssc registered [ 5478.411769] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5497.349954] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 12:54:03 (1787849643) [ 5518.012560] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_hostid [ 5532.995930] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 5577.510736] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5577.556976] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5578.850591] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5578.883754] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5578.958805] Lustre: lustre-MDT0000: new disk, initializing [ 5579.074918] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5579.095426] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5584.569595] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 5597.456533] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5597.662697] Lustre: 166927:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5597.743813] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5597.753256] Lustre: Skipped 1 previous similar message [ 5597.853651] Lustre: lustre-MDT0001: new disk, initializing [ 5597.969688] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5598.036118] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5598.053362] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5603.724593] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 5615.873614] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5622.982098] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5631.410826] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5631.793817] Lustre: lustre-OST0000: new disk, initializing [ 5631.797513] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5631.805933] Lustre: 168898:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5631.943982] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5640.508477] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 5640.789282] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5640.806905] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5640.995244] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5652.460645] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5666.793738] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5666.804922] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5666.829621] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5670.371327] Lustre: server umount lustre-OST0000 complete [ 5677.026080] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5677.037185] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5677.050892] Lustre: Skipped 2 previous similar messages [ 5677.062598] Lustre: Skipped 2 previous similar messages [ 5682.031374] Lustre: server umount lustre-MDT0000 complete [ 5686.211251] LustreError: 166918:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787849834 with bad export cookie 3826547055608674947 [ 5686.221602] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5686.227798] LustreError: 166918:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5686.787926] Lustre: server umount lustre-MDT0001 complete [ 5701.796334] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [ 5705.633547] Key type lgssc unregistered [ 5705.950315] LNet: 170808:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5705.969941] LNetError: 170808:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5706.005046] LNet: Removed LNI 192.168.202.157@tcp [ 5707.020105] Key type .llcrypt unregistered [ 5707.021808] Key type ._llcrypt unregistered [ 5736.774926] Key type ._llcrypt registered [ 5736.776648] Key type .llcrypt registered [ 5736.885674] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 5738.411924] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5738.500963] alg: No test for adler32 (adler32-zlib) [ 5739.585474] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [ 5739.985291] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [ 5741.715268] Key type lgssc registered [ 5743.399383] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5752.719124] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5752.749957] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5754.424199] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5760.236425] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 5764.578099] LustreError: 172224:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5764.589079] LustreError: 172224:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 2 previous similar messages [ 5769.696459] LustreError: 172225:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5770.509227] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5770.885680] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5776.066349] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 5784.946210] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5791.564866] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5800.004418] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5800.397817] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5808.349234] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 5818.275455] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5823.922026] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5830.240939] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5830.275879] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5830.296675] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5830.311102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.349915] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5831.625151] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5831.653491] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5831.665242] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5831.673033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.690517] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5833.094057] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5833.104209] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5833.121616] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5833.126972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.135074] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5834.124557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.428114] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5835.438914] Lustre: Skipped 1 previous similar message [ 5835.464228] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5835.477094] Lustre: Skipped 1 previous similar message [ 5835.480449] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5835.485611] Lustre: Skipped 1 previous similar message [ 5835.489235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.499244] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5835.506939] Lustre: Skipped 1 previous similar message [ 5836.917986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.395129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.409860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.745272] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5840.767520] Lustre: Skipped 3 previous similar messages [ 5840.850917] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5840.888721] Lustre: Skipped 3 previous similar messages [ 5840.916083] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5840.928282] Lustre: Skipped 3 previous similar messages [ 5840.941073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.959236] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5840.982910] Lustre: Skipped 3 previous similar messages [ 5842.521844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.094023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.576361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.628838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.526987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.454437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.805147] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5849.808178] Lustre: Skipped 6 previous similar messages [ 5849.819651] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5849.827633] Lustre: Skipped 6 previous similar messages [ 5849.831195] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5849.834025] Lustre: Skipped 6 previous similar messages [ 5849.836958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.849081] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5849.854458] Lustre: Skipped 6 previous similar messages [ 5851.055469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.920767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.047804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.865196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.985652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.960980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.315213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.674701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.868301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.178894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.446296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.600629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.688938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.702672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.114705] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5867.123349] Lustre: Skipped 14 previous similar messages [ 5867.151700] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5867.175221] Lustre: Skipped 14 previous similar messages [ 5867.186075] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5867.193681] Lustre: Skipped 14 previous similar messages [ 5867.199219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.208878] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5867.211313] Lustre: Skipped 14 previous similar messages [ 5868.369715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.654713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.481643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.267518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.945602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.908276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.382712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.792133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.312847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.523550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.717562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.573220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.686809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.668681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.891814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.771665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.750074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.038985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.070194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.446946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.408631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.627719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.577066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.210919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.140123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.535698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.815980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.565866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.509370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.844950] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5899.854286] Lustre: Skipped 29 previous similar messages [ 5899.877479] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5899.882472] Lustre: Skipped 29 previous similar messages [ 5899.884506] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5899.888276] Lustre: Skipped 29 previous similar messages [ 5899.890821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.898385] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5899.901120] Lustre: Skipped 29 previous similar messages [ 5900.977394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.005535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.134409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.021687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.052531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.190660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.247210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.251042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.140719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.172952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.061740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.167851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.195775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.384739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.452158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.276457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.206830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.095730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.103338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.128289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.188324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.127132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.300657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.430536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.388209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.414566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.401472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.463091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.306764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.365405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.394741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.333631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.407838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.364583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.539240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.511571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.704227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.647357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.943543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.004644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.099760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.153305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.306603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.330576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.409219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.337172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.390251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.555424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.582586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.552928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.641619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.778034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.061905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.645126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.927597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.224785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.301627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.520073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.460613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.681982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.903230] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5964.908835] Lustre: Skipped 60 previous similar messages [ 5964.934344] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5964.942171] Lustre: Skipped 60 previous similar messages [ 5964.947029] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5964.952782] Lustre: Skipped 60 previous similar messages [ 5964.956072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.969255] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5964.984172] Lustre: Skipped 60 previous similar messages [ 5966.110186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.048297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.058110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.152461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.082258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.022973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.045393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.095341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.132544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.867868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.960096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.142570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.188588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.251462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.594274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.699552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.840570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.961625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.067266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.204831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.326065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.483479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.594708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.626265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.184654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.285198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.569379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.592118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.826124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.034773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.146654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.208880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.275045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.521946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.465481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.414746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.546638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.532440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.579463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.785600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.851173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.983813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.316642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.658609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.731390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.940077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.335396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.537060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.645122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.820035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.947314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.888332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.788124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.651133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.678373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.552197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.576581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.537778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.536509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.399443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.329703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.253453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.218445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.343941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.472237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.432647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.688897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.701490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.879032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.005388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.981825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.126733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.367108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.579038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.624385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.802416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.854283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.700583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.647721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.511575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.382558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.334114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.239539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.197730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.232459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.095271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.081330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.900758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.154031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.322500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.234253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.424075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.489910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.670767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.792377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.827480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.883671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.963811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.127543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.158227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.303291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.351158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.431487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.497903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.405150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.382948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.235613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.347846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.538120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.464881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.390415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.389271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.283211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.422603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.285433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.482301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.782360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.838410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.472057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.243122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.915526] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6092.921318] Lustre: Skipped 120 previous similar messages [ 6092.934832] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6092.939106] Lustre: Skipped 120 previous similar messages [ 6092.941761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.794958] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6093.797747] Lustre: Skipped 121 previous similar messages [ 6093.800948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.806446] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6093.809930] Lustre: Skipped 121 previous similar messages [ 6094.551792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.486614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.033207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.483062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.881321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.013420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.698624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.479404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.159410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.483439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.880212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.953734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.453166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.315790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.222223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.224319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.208814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.164498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.373773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.429489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.536632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.909576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.013873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.939315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.993207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.268309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.500582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.577556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.846384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.750243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.037742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.166340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.337479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.947634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.860711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.390332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.134827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.139786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.573599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.512931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.427326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.544834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.651261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.217079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.434490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.155768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.888393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.744199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.677989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.776917] LustreError: 172244:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6150.793311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.797507] LustreError: 172244:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6151.698880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.912814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.125245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.337767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.478693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.443159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.614615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.884782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.132036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.057158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.053764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.881514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.835351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.375802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.460035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.357964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.784183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.561200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.196513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.112535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.416335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.240413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.400248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.006593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.295748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.635445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.111538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.427184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.050750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.976060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.169695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.254424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.323673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.472220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.887826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.196308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.473197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.481648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.435139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.478734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.482589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.652715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.087925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.753276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.230977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.759886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.754047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.637583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.496359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.424628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.460306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.498213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.494087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.617443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.898559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.990501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.074154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.301912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.413101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.299370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.265748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.072193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.389125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.365708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.367066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.400253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.404564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.602971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.770248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.733756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.733851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.860073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.831708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.690751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.835757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.822035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.681078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.087467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.301154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.323449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.392573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.534933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.859268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.765468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.852943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.150506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.191601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.572995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.140822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.749171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.558315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.135475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.564512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.711745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.800548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.219551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.637370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.935237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.812967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.789052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.707053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.725362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.651796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.799857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.856191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.796934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.046872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.125371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.171125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.101636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.049356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.898957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.781615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.725605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.675168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.608819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.650479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.880748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.035888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.213102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.287722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.283740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.512892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.651916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.766247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.980390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.222450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.327697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.523919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.413387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.543449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.713996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.857505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.251055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.331550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.529605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.612325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.761602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.979440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.216820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.097323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.232848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.265806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.293946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.189427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.554638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.888906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.209800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.408399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.697587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.847359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.136533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.289594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.387669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.894814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.204745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.688790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.053606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.213706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.432148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.514785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.637718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.835501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.853205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.900843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.882889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.005540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.953558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.056701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.272341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.344484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.430031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.713966] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6349.718147] Lustre: Skipped 223 previous similar messages [ 6349.744741] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6349.754842] Lustre: Skipped 223 previous similar messages [ 6349.759976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.833548] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6350.845942] Lustre: Skipped 223 previous similar messages [ 6350.849388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.864939] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6350.887888] Lustre: Skipped 223 previous similar messages [ 6351.683478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.755924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.982447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.756319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.903834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.938942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.214858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.063262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.087971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.149465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.160918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.170494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.373746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.193732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.320281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.311512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.424325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.379538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.392781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.660327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.891333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.210412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.179491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.453429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.773517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.880589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.851070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.877200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.797822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.920628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.815553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.796946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.672970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.767177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.759313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.815078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.822140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.790090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.523459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.585377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.709174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.980947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.984029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.087940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.261792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.033758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.133526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.015874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.487462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.548880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.673469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.563402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.672765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.663207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.976743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.336323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.451466] LustreError: 172244:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6411.469375] LustreError: 172244:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6411.482029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.015612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.840696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.939596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.702398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.506483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.125856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.026888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.389932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.650551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.783995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.888926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.011257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.216849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.539070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.083890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.178809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.099250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.187392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.111877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.725647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.849190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.949059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.886952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.247834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.308996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.338704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.898410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.991709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.206152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.288116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.471345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.498619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.270806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.456301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.752496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.874846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.161022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.576300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.598963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.897672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.219751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.386224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.703737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.966985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.865442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.294427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.327528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.255207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.238644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.058966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.113342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.972941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.109411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.452952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.693284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.043591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.371911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.480295] LustreError: 172244:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6477.504735] LustreError: 172244:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6477.526452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.536831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.674698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.718960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.606051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.874529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.775979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.781592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.903336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.176561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.101575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.289492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.378030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.447943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.639481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.627774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.679619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.866528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.114209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.185457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.296103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.522653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.513407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.612619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.667389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.645932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.972309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.264056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.270608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.347182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.207571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.277953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.266845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.067952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.083060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.057542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.163675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.260290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.080486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.103030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.905972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.020096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.060717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.181612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.335779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.308209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6526.441029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.371651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.644158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6529.637636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6530.744380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.793405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.762095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.669083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6534.659894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.484775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.624065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.651042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6538.915787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.860792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.111355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.363592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.536651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.957073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.102665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.245081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.563574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.577657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.984570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6552.298099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.717726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.698690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.793971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.754387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.966633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.949305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.102850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.273459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.286973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.216918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.955779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.254470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.480845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.621741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.706641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.665889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.592032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.740918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.721260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.924285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.945501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.502790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.105633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.470898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.705244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.375341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.955851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.925692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.202438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.453137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.669643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.319238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6592.794525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.221974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.532458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.682152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.563821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.291610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.205729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.500902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.578755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.394599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.680994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.495379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6606.089321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6606.846534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6607.621758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.778993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.780701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.093985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.441816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.629207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.973709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6615.973744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6617.461958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.596498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6619.830615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.770888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6621.688081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.458464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6623.596278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.715308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6625.697367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6626.797141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.758226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.882957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.826206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6630.928705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6632.108107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6633.359704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6634.682562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6635.761436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6636.960169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6638.017126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.360048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6640.335618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6641.455211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6642.423424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6643.787929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6645.050388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6646.166326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.119706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6648.257169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6649.525260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.632258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6651.787364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6653.162803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.081290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.366299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6656.373628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6657.578685] LustreError: 172244:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6657.589685] LustreError: 172244:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6657.602796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6658.533040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.678135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6660.794831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.945298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6663.139377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6664.401126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6665.512324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6666.487187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6667.608361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6668.491349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6669.633226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6670.688347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6672.195178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6673.555511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6674.760401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6676.037455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6677.264403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6679.069656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6680.040921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6680.949494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6681.888793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6683.152485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6684.259157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6685.728650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6687.558126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6689.657765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6691.673372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6692.887685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6693.974428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6695.209354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6696.480578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6697.682812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6698.828557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6700.090618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6701.420229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6702.443216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6703.579940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6705.073834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6706.271088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6707.807426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6708.983085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6710.053803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6711.105366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6712.401977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6713.706825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6714.824941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6715.991934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6716.883544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6717.878739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6719.082283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6720.358664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6721.415323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6722.521901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6723.794581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6724.993818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6725.904411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6727.043400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6728.192591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6729.374318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6730.573910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6731.794727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6732.833983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6734.161459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6735.552630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6736.895414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6737.826919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6739.054284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6740.325985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6741.219341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6742.479681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6743.433869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6744.563478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6745.529450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6746.619963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6747.951845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6749.076362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6750.041951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6751.134942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6752.106743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6753.200368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6754.301559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6755.358604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6756.426829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6757.257566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6758.370776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6759.404497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6760.458419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6761.358737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6762.425171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6763.622604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6764.793984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6766.064418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6767.472155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6768.799843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6769.862982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6770.980985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6772.207742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6773.440561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6774.627415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6775.522340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6776.734154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6777.842721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6779.137892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6780.152555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6781.296207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6782.487281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6783.671080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6784.996391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6786.172896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6787.363854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6788.231981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6789.240428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6790.271536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6791.261562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6792.023745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6793.042106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6794.125271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6795.596306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6796.928398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6798.031809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6799.201564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6800.375678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6801.265383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6802.017184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6802.996109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6804.089558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6805.156541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6806.479495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6807.898467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6809.118446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6809.919550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6811.357549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6813.200956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6814.261834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6815.583580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6816.992182] LustreError: 172244:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6817.010546] LustreError: 172244:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6817.017995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6818.468936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6819.361374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6820.143885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6821.265344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6822.485467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6823.535440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6824.581953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6826.080400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6827.404072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6829.117671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6830.558191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6831.847254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6833.006847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6833.933479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6835.320248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6836.837152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6838.068379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6839.475360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6840.387498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6841.348648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6842.371121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6843.495672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6844.677046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6845.706760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6846.451463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6847.142304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6848.017704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6848.912488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6850.107271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6851.303459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6852.627789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6853.605461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6854.680656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6855.676612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6856.826315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6857.878237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6859.291621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6860.049142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6860.992367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6861.790609] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6861.799054] Lustre: Skipped 451 previous similar messages [ 6861.814295] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6861.818659] Lustre: Skipped 451 previous similar messages [ 6861.822192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6862.638821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6863.653952] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6863.661194] Lustre: Skipped 452 previous similar messages [ 6863.664541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6863.671691] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6863.674988] Lustre: Skipped 452 previous similar messages [ 6864.585301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6865.635884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6867.025780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6868.133226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6868.927395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6869.619072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6870.468805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6871.356488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6872.285063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6873.340334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6874.363438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6875.360444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6876.265500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6877.311853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6878.112460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6879.097857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6880.055952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6881.095536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6882.110421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6883.103807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6883.836236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6884.909701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6885.796354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6886.789293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6887.812478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6888.876430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6889.729094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6890.771914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6891.674866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6892.788809] LustreError: 172244:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6892.797056] LustreError: 172244:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6892.809356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6893.752727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6894.811647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6895.643444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6896.237675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6896.839555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6897.595719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6898.382635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6899.346492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6900.177377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6901.312827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6902.117320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6903.017138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6903.909470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6904.937684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6905.758878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6906.866346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6907.855992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6908.922484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6910.172915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6911.236337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6912.241863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6913.367232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6914.043700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6914.942508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6915.824229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6916.767727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6917.451380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6918.348555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6919.171609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6920.217142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6921.067186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6921.969060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6922.732415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6923.472494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6924.655923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6925.987390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6927.455948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6928.747625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6930.172519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6930.979694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6932.236609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6933.563478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6934.566129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6935.575615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6936.767401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6937.770454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6938.771158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6939.767884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6940.821752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7386.374260] LustreError: 174048:0:(ofd_dev.c:1783:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7386.384634] LustreError: 172244:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 8109.555659] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8109.563188] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8109.594110] Lustre: Skipped 81 previous similar messages [ 8109.607152] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8114.145907] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8114.188780] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8115.894638] Lustre: server umount lustre-OST0000 complete [ 8120.801737] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8120.823350] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8120.847452] Lustre: Skipped 1 previous similar message [ 8124.897054] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8127.785466] Lustre: server umount lustre-MDT0000 complete [ 8130.018615] LustreError: 172954:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8130.031375] LustreError: 172954:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [ 8134.225412] LustreError: 172206:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787852282 with bad export cookie 18349112015230434810 [ 8134.232612] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8134.259406] LustreError: 172206:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8135.386544] Lustre: server umount lustre-MDT0001 complete [ 8147.994495] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [ 8151.213875] Key type lgssc unregistered [ 8151.441888] LNet: 227545:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8151.459303] LNetError: 227545:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8151.479739] LNet: Removed LNI 192.168.202.157@tcp [ 8152.402564] Key type .llcrypt unregistered [ 8152.404261] Key type ._llcrypt unregistered [ 8185.539679] Key type ._llcrypt registered [ 8185.546048] Key type .llcrypt registered [ 8185.878773] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 8187.831986] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8187.865712] alg: No test for adler32 (adler32-zlib) [ 8189.941651] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [ 8190.406367] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [ 8192.207177] Key type lgssc registered [ 8194.872306] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8206.071380] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8206.092739] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8207.771094] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8213.786621] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 8217.066039] LustreError: 228969:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8217.088115] LustreError: 228969:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 2 previous similar messages [ 8222.180976] LustreError: 228968:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8223.926043] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8224.342889] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8229.752393] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 8238.480025] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8246.233250] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8256.091400] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8256.490762] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8263.036870] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 8271.626505] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8278.031984] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 8281.759654] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 8284.048363] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8284.063490] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 8284.068418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8284.081625] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8284.090248] LustreError: 228988:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 8288.229119] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8288.231082] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8288.246401] Lustre: Skipped 1 previous similar message [ 8288.260664] Lustre: Skipped 1 previous similar message [ 8292.618749] Lustre: server umount lustre-OST0000 complete [ 8304.153679] LustreError: 228988:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 8327.888329] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8328.164343] LustreError: 232176:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8328.195515] LustreError: 232176:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [ 8328.381433] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8333.802481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8333.830439] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8333.835404] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8333.855923] Lustre: Skipped 1 previous similar message [ 8336.660739] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 8345.314254] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8346.892381] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 8353.374944] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9998921ce800.ost_server_uuid 50 [ 8355.533920] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9998921ce800.ost_server_uuid in IDLE state after 0 sec [ 8360.930115] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8360.936551] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8360.943350] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8360.947160] Lustre: Skipped 1 previous similar message [ 8364.515861] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8364.520021] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8364.526058] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8366.577976] Lustre: server umount lustre-OST0000 complete [ 8374.753623] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8374.780389] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8374.792204] Lustre: Skipped 2 previous similar messages [ 8379.565488] Lustre: server umount lustre-MDT0000 complete [ 8379.886281] LustreError: 228969:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8379.913370] LustreError: 228969:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [ 8383.996052] LustreError: 229697:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8384.005861] LustreError: 229697:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [ 8385.362387] LustreError: 228948:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787852533 with bad export cookie 12788330050685256668 [ 8385.379141] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8386.074970] Lustre: server umount lustre-MDT0001 complete [ 8400.679313] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [ 8404.299234] Key type lgssc unregistered [ 8404.613987] LNet: 234180:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8404.634749] LNetError: 234180:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8404.661041] LNet: Removed LNI 192.168.202.157@tcp [ 8405.889101] Key type .llcrypt unregistered [ 8405.889975] Key type ._llcrypt unregistered [ 8450.372478] Key type ._llcrypt registered [ 8450.380734] Key type .llcrypt registered [ 8450.591714] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 8451.927396] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8452.043487] alg: No test for adler32 (adler32-zlib) [ 8453.288637] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [ 8453.578220] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [ 8455.295521] Key type lgssc registered [ 8456.632912] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8469.590914] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8469.619060] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8471.392719] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8477.566960] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 8481.762369] LustreError: 235913:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8481.782840] LustreError: 235913:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 2 previous similar messages [ 8486.881289] LustreError: 235914:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8487.521038] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8487.754912] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8492.181221] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 8495.568548] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8505.795531] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8506.237483] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8514.551408] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 8516.486515] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 8525.795362] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8526.085323] Lustre: lustre-OST0001: new disk, initializing [ 8526.090573] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8526.106746] Lustre: 238225:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8526.372097] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8534.217523] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8534.240230] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8534.432022] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8535.331408] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 8549.108390] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8556.035707] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8569.824987] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8569.827830] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8569.861459] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8570.339246] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8570.369312] Lustre: Skipped 2 previous similar messages [ 8570.392047] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8570.394278] Lustre: Skipped 3 previous similar messages [ 8575.458586] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8575.481611] Lustre: Skipped 3 previous similar messages [ 8576.032841] Lustre: server umount lustre-MDT0000 complete [ 8580.396601] LustreError: 235896:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787852728 with bad export cookie 11809663680998578494 [ 8580.401718] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8580.417532] LustreError: 235896:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8580.577362] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8580.579672] LustreError: 235908:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8580.600605] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8580.633197] LustreError: 235908:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [ 8581.245996] Lustre: server umount lustre-MDT0001 complete [ 8587.249452] Lustre: server umount lustre-OST0000 complete [ 8591.551077] Lustre: server umount lustre-OST0001 complete [ 8616.404266] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 8628.170380] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8628.594041] LustreError: 241329:0:(ldlm_lib.c:1212:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8628.678890] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8633.823815] LustreError: 241330:0:(ldlm_lib.c:1212:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8634.352656] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 8643.043906] LustreError: 241329:0:(ldlm_lib.c:1212:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8643.084122] LustreError: 241329:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [ 8644.907363] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8645.583638] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8652.079900] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 8656.833734] Lustre: 242473:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8665.454995] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8666.162924] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8671.282948] LustreError: 242828:0:(ldlm_lib.c:1212:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8671.301941] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 8673.355429] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 8674.080147] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 8684.960706] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8691.643941] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 8699.814430] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8705.114907] Lustre: 244345:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8721.381297] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8721.396660] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8721.400905] Lustre: Skipped 3 previous similar messages [ 8721.419188] Lustre: Skipped 2 previous similar messages [ 8727.598685] Lustre: server umount mylustre-MDT0000 complete [ 8731.625758] LustreError: 242063:0:(ldlm_lib.c:1212:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8731.675390] LustreError: 242063:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 9 previous similar messages [ 8732.408582] LustreError: 241304:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787852880 with bad export cookie 11809663680998581357 [ 8732.428238] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8732.433552] LustreError: 241304:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8733.174435] Lustre: server umount mylustre-MDT0001 complete [ 8747.532865] Lustre: server umount mylustre-OST0000 complete [ 8761.712883] Lustre: server umount mylustre-OST0001 complete [ 8788.099680] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 8798.795797] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8799.238818] LustreError: 246657:0:(ldlm_lib.c:1212:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8799.336171] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8799.347033] Lustre: Skipped 1 previous similar message [ 8804.096454] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 8813.703804] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8819.207950] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 8822.444226] Lustre: 247801:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8829.523841] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8837.644732] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 8839.235271] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 8845.324850] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 8847.427762] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8856.114864] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 8864.557827] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8868.452891] Lustre: 249674:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8878.585204] Lustre: tfs-MDT0000-lwp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8878.589705] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8878.602684] Lustre: Skipped 3 previous similar messages [ 8878.633256] Lustre: Skipped 8 previous similar messages [ 8883.834358] Lustre: server umount tfs-MDT0000 complete [ 8888.239218] LustreError: 246633:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787853036 with bad export cookie 11809663680998583310 [ 8888.260686] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8888.705470] Lustre: server umount tfs-MDT0001 complete [ 8904.223185] Lustre: 235178:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787853036/real 1787853036] req@ffff8cc7567692c0 x1874699329169664/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787853052 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8904.271375] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8904.294866] Lustre: Skipped 1 previous similar message [ 8904.529975] Lustre: server umount tfs-OST0000 complete [ 8908.780411] Lustre: server umount tfs-OST0001 complete [ 8935.777487] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 8949.009755] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8949.565984] LustreError: 251925:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8949.584720] LustreError: 251925:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 10 previous similar messages [ 8949.676116] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8949.685988] Lustre: Skipped 3 previous similar messages [ 8955.832561] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 8966.092261] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8971.983311] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 8975.538710] Lustre: 253067:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8985.019409] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8991.758089] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 8993.678223] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 8995.855199] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 9003.809593] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9012.468057] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 9021.613711] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9026.616521] Lustre: 254939:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 9035.317739] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 13:53:01 (1787853181) [ 9053.670640] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9053.675594] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9053.693420] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9060.109375] Lustre: server umount lustre-MDT0000 complete [ 9064.452784] LustreError: 254249:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787853212 with bad export cookie 11809663680998584997 [ 9064.460405] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9064.469114] LustreError: 254249:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 9071.077180] Lustre: server umount lustre-MDT0001 complete [ 9081.975554] Lustre: server umount lustre-OST0000 complete [ 9093.022755] Lustre: server umount lustre-OST0001 complete [ 9101.756289] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_hostid [ 9113.249767] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 9157.239874] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9157.673227] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9157.734835] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9157.847741] Lustre: lustre-MDT0000: new disk, initializing [ 9158.054170] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 9158.062567] Lustre: Skipped 3 previous similar messages [ 9158.123979] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9164.575554] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 9177.579414] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9177.753828] Lustre: 258126:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9177.810762] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9177.814834] Lustre: Skipped 1 previous similar message [ 9177.911852] Lustre: lustre-MDT0001: new disk, initializing [ 9178.162611] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9178.172274] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9184.087504] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 9196.687560] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9197.135355] Lustre: lustre-OST0000: new disk, initializing [ 9197.138846] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9197.143343] Lustre: 259711:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9198.488141] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9198.501815] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9198.698246] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9205.070500] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 9218.663171] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9218.973282] Lustre: lustre-OST0001: new disk, initializing [ 9218.977262] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9218.992156] Lustre: 260581:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9220.697415] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9220.709907] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9220.853688] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9227.393690] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 9243.931322] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 13:56:29 (1787853389) [ 9265.168650] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 13:56:50 (1787853410) [ 9282.531747] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9282.558048] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9282.591022] Lustre: Skipped 6 previous similar messages [ 9282.595950] Lustre: Skipped 8 previous similar messages [ 9288.549380] Lustre: server umount lustre-OST0000 complete [ 9292.772322] LustreError: 259694:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9292.815651] LustreError: 259694:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 17 previous similar messages [ 9295.845215] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9299.650474] Lustre: server umount lustre-MDT0000 complete [ 9305.646198] LustreError: 258121:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787853453 with bad export cookie 11809663680998586558 [ 9305.653351] LustreError: 258121:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9305.670219] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9312.106882] Lustre: server umount lustre-MDT0001 complete [ 9329.450213] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [ 9402.972558] Lustre: server umount lustre-OST0001 complete [ 9412.070896] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_hostid [ 9433.073412] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 9478.551371] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9479.065923] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9479.123711] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9479.238307] Lustre: lustre-MDT0000: new disk, initializing [ 9479.400685] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9479.403777] Lustre: Skipped 3 previous similar messages [ 9479.465266] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9485.456615] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 9497.503263] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9497.614478] Lustre: 265435:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9497.682892] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9497.690084] Lustre: Skipped 1 previous similar message [ 9497.768599] Lustre: lustre-MDT0001: new disk, initializing [ 9497.917893] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9497.934967] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9502.945364] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 9514.749071] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9521.252902] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9529.341443] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9529.684725] Lustre: lustre-OST0000: new disk, initializing [ 9529.693065] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9529.701456] Lustre: 267414:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9530.984892] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9530.997825] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9531.107846] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9536.383231] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 9547.569316] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9556.965651] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9556.979312] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9556.983044] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9556.991935] Lustre: Skipped 5 previous similar messages [ 9557.011647] Lustre: Skipped 10 previous similar messages [ 9562.584085] Lustre: server umount lustre-OST0000 complete [ 9573.355606] Lustre: server umount lustre-MDT0000 complete [ 9578.095887] LustreError: 265429:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787853726 with bad export cookie 11809663680998588028 [ 9578.109022] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9578.117624] LustreError: 265429:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 9578.565964] Lustre: server umount lustre-MDT0001 complete [ 9589.621580] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [ 9592.856714] Key type lgssc unregistered [ 9593.271320] LNet: 269178:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9593.285612] LNetError: 269178:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9593.305149] LNet: Removed LNI 192.168.202.157@tcp [ 9594.103172] Key type .llcrypt unregistered [ 9594.121831] Key type ._llcrypt unregistered [ 9625.200153] Key type ._llcrypt registered [ 9625.202478] Key type .llcrypt registered [ 9625.301221] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 9627.010067] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9627.131396] alg: No test for adler32 (adler32-zlib) [ 9628.306563] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [ 9628.615143] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [ 9630.313080] Key type lgssc registered [ 9631.595785] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9640.181350] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9640.198849] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9641.798544] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9647.150450] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 9651.169904] LustreError: 270644:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9651.190773] LustreError: 270644:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 2 previous similar messages [ 9656.300294] LustreError: 270643:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9657.540171] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9658.023189] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9663.578959] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 9671.763370] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9678.974650] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9686.130762] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9686.411202] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9692.577061] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 9701.685592] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9702.705718] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9709.039657] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9709.041373] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9709.088434] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9711.078352] Lustre: server umount lustre-OST0000 complete [ 9716.197839] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9716.212169] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9721.313349] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9721.322051] Lustre: Skipped 2 previous similar messages [ 9721.462553] Lustre: server umount lustre-MDT0000 complete [ 9726.123202] LustreError: 270625:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787853874 with bad export cookie 9782928997605426921 [ 9726.125262] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9726.132146] LustreError: 270625:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9726.477305] Lustre: server umount lustre-MDT0001 complete [ 9740.045315] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [ 9743.264208] Key type lgssc unregistered [ 9743.656474] LNet: 274232:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9743.692536] LNetError: 274232:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9743.715081] LNet: Removed LNI 192.168.202.157@tcp [ 9744.717079] Key type .llcrypt unregistered [ 9744.725069] Key type ._llcrypt unregistered [ 9763.101843] Key type ._llcrypt registered [ 9763.103538] Key type .llcrypt registered [ 9763.268478] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 9764.690696] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9764.770494] alg: No test for adler32 (adler32-zlib) [ 9765.990914] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [ 9766.333561] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [ 9768.047178] Key type lgssc registered [ 9769.908425] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9781.048236] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9781.074524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9796.704596] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9801.823845] LustreError: 275427:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.157@tcp: failed processing log, type 4: rc = -110 [ 9828.511694] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9834.620297] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [ 9843.147396] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9845.729258] Lustre: server umount lustre-OST0000 complete [ 9862.097953] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [ 9865.139642] Key type lgssc unregistered [ 9865.439533] LNet: 277061:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9865.448137] LNetError: 277061:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9866.473654] LNet: Removed LNI 192.168.202.157@tcp [ 9867.299059] Key type .llcrypt unregistered [ 9867.300746] Key type ._llcrypt unregistered [ 9911.875717] Key type ._llcrypt registered [ 9911.877977] Key type .llcrypt registered [ 9912.017763] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [ 9913.632357] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9913.658270] alg: No test for adler32 (adler32-zlib) [ 9914.817978] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [ 9915.132074] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [ 9916.839449] Key type lgssc registered [ 9918.242389] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9933.753452] loop0: detected capacity change from 0 to 204800000 [ 9933.982379] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9933.996788] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9934.379538] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9935.740271] loop0: detected capacity change from 0 to 204800000 [ 9935.968199] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9935.982476] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9936.314536] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9937.501842] loop0: detected capacity change from 0 to 204800000 [ 9937.783914] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9937.798201] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9937.930530] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9939.190280] loop0: detected capacity change from 0 to 204800000 [ 9939.357668] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9939.373716] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9939.522351] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9940.672449] loop0: detected capacity change from 0 to 204800000 [ 9940.701756] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9945.363169] loop0: detected capacity change from 0 to 204800000 [ 9945.398922] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9949.662450] loop0: detected capacity change from 0 to 204800000 [ 9949.691289] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9953.825837] loop0: detected capacity change from 0 to 204800000 [ 9953.866701] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9958.698848] loop0: detected capacity change from 0 to 204800000 [ 9958.831579] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9958.925465] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9958.982488] Lustre: 280479:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9958.988438] Lustre: 280479:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9958.998611] Lustre: 280479:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9959.005548] Lustre: 280479:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9959.012522] Lustre: 280479:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9959.019624] Lustre: 280479:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9959.026603] Lustre: 280479:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9959.035276] Lustre: 280479:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9959.042437] Lustre: 280479:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9959.050924] Lustre: 280479:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9959.076346] Lustre: 280479:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9959.100992] Lustre: 280479:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9959.114721] Lustre: 280479:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9959.125235] Lustre: 280479:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9965.739714] Lustre: server umount MGS complete [ 9966.642216] loop0: detected capacity change from 0 to 204800000 [ 9966.790605] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9967.396853] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9967.454228] Lustre: lustre-MDT0000: Aborting client recovery [ 9968.677814] loop1: detected capacity change from 0 to 204800000 [ 9968.863658] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9968.991480] Lustre: 280915:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9969.012033] Lustre: 280915:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9969.045706] Lustre: 280915:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9969.057811] Lustre: 280915:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9969.081637] Lustre: 280915:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9969.119484] Lustre: 280915:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9969.128448] Lustre: 280915:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9969.148623] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9969.464884] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9969.580201] Lustre: lustre-MDT0001: Aborting client recovery [ 9970.693827] loop2: detected capacity change from 0 to 204800000 [ 9970.797406] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9970.893668] Lustre: 280993:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9970.915748] Lustre: 280993:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9970.938591] Lustre: 280993:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9970.954536] Lustre: 280993:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9970.975855] Lustre: 280993:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9970.997849] Lustre: 280993:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9971.014548] Lustre: 280993:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9971.237825] Lustre: Found index 0 for lustre-OST0000, updating log [ 9971.286253] Lustre: 281012:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9971.350211] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9971.371879] Lustre: lustre-OST0000: Aborting recovery [ 9972.366903] loop3: detected capacity change from 0 to 204800000 [ 9972.586115] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9972.728929] Lustre: 281077:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9972.736679] Lustre: 281077:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9972.751769] Lustre: 281077:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9972.771597] Lustre: 281077:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9972.785679] Lustre: 281077:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9972.803640] Lustre: 281077:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9972.832259] Lustre: 281077:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9972.858815] Lustre: Found index 1 for lustre-OST0001, updating log [ 9972.919461] Lustre: 281084:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9973.060296] Lustre: lustre-OST0001: Aborting recovery [ 9977.129912] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9977.138781] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9977.163787] LustreError: 280858:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 9977.216456] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9977.276976] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9979.235804] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9979.235926] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9981.579172] LustreError: 280825:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 498us: rc = -2 [ 9982.089492] LustreError: 280823:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 602us: rc = -2 [ 9982.112296] LustreError: 280823:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 20 previous similar messages [ 9995.232264] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9995.251672] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9995.263615] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9999.841519] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9999.844657] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9999.853558] Lustre: Skipped 2 previous similar messages [ 9999.860491] Lustre: Skipped 3 previous similar messages [10001.294712] Lustre: server umount lustre-MDT0000 complete [10002.297363] LustreError: 280809:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787854150 with bad export cookie 171923815011755639 [10002.340756] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10004.959799] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10004.984459] LustreError: 281363:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10004.985788] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10005.000170] LustreError: 281363:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 3 previous similar messages [10005.011806] Lustre: Skipped 1 previous similar message [10007.968784] LustreError: 281302:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10007.969868] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10007.984053] LustreError: 281302:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [10008.011388] Lustre: Skipped 1 previous similar message [10008.876487] Lustre: server umount lustre-MDT0001 complete [10015.970542] Lustre: server umount lustre-OST0000 complete [10031.071460] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10031.312189] Lustre: server umount lustre-OST0001 complete [10039.946672] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 14:09:46 (1787854186) [10068.961362] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_hostid [10077.991354] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [10115.402387] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10115.605935] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10115.642521] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10115.718449] Lustre: lustre-MDT0000: new disk, initializing [10115.774831] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10115.777946] Lustre: Skipped 1 previous similar message [10115.792268] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10120.656216] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [10131.273763] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10131.421197] Lustre: 285154:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10131.475412] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10131.479414] Lustre: Skipped 1 previous similar message [10131.570602] Lustre: lustre-MDT0001: new disk, initializing [10131.690195] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10131.728147] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10131.738802] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10136.334546] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [10146.963446] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10152.750051] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10160.144696] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10160.470221] Lustre: lustre-OST0000: new disk, initializing [10160.474405] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10160.482243] Lustre: 287122:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10160.577238] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10167.845182] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [10170.390378] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10170.400377] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10170.505160] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10178.370773] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10183.748919] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10199.084096] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10205.027154] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10257.378644] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10257.385149] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10257.395613] Lustre: Skipped 1 previous similar message [10257.400367] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10262.570947] Lustre: server umount lustre-OST0000 complete [10268.128604] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10268.141134] Lustre: Skipped 1 previous similar message [10268.153037] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10268.166628] Lustre: Skipped 2 previous similar messages [10272.771847] Lustre: server umount lustre-MDT0000 complete [10273.248604] LustreError: 286009:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10273.268320] LustreError: 286009:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [10276.710392] LustreError: 287979:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787854424 with bad export cookie 171923815011758362 [10276.713083] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10276.721856] LustreError: 287979:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [10277.125238] Lustre: server umount lustre-MDT0001 complete [10285.684097] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10290.575522] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [10295.926688] Lustre: server umount MGS complete [10307.859913] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10308.330409] LustreError: 291086:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10308.367940] LustreError: 291099:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10308.452411] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10312.432616] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [10313.703319] LustreError: 291087:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10321.324939] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10326.172086] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [10333.230802] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10338.627578] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10344.368160] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10344.702468] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10344.707074] Lustre: Skipped 1 previous similar message [10351.221030] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [10359.199234] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10393.056232] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10393.061648] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10393.080545] Lustre: Skipped 1 previous similar message [10393.088993] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10398.475460] Lustre: server umount lustre-OST0000 complete [10403.814067] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10403.821552] LustreError: Skipped 1 previous similar message [10409.191880] Lustre: server umount lustre-MDT0000 complete [10409.442294] LustreError: 291087:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10409.450262] LustreError: 291087:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 2 previous similar messages [10412.763817] LustreError: 293812:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787854560 with bad export cookie 171923815011760203 [10412.769910] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10412.770641] LustreError: 293812:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10413.177726] Lustre: server umount lustre-MDT0001 complete [10424.333668] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [10426.948419] Key type lgssc unregistered [10427.288699] LNet: 295035:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10427.297568] LNetError: 295035:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10427.316484] LNet: Removed LNI 192.168.202.157@tcp [10428.203866] Key type .llcrypt unregistered [10428.205549] Key type ._llcrypt unregistered [10466.986834] Key type ._llcrypt registered [10466.988864] Key type .llcrypt registered [10467.168592] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_hostid [10481.994095] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [10482.965554] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10483.021164] alg: No test for adler32 (adler32-zlib) [10484.088402] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [10484.264141] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [10485.927269] Key type lgssc registered [10487.109439] Lustre: Echo OBD driver; http://www.lustre.org/ [10526.051880] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10526.108111] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10527.447295] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10527.531883] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10527.668348] Lustre: lustre-MDT0000: new disk, initializing [10527.822072] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10527.870867] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10532.610248] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [10543.643359] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10543.799128] Lustre: 298996:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10543.844822] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10543.850873] Lustre: Skipped 1 previous similar message [10543.939862] Lustre: lustre-MDT0001: new disk, initializing [10544.022186] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10544.073018] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10544.092331] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10548.378720] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [10557.465675] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10563.320517] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10570.259574] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10570.538821] Lustre: lustre-OST0000: new disk, initializing [10570.546498] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10570.564489] Lustre: 300964:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10570.674349] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10577.275911] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [10578.025423] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10578.043633] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10578.178956] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10587.251704] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10594.448292] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10604.513068] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10610.660073] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10660.333719] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10660.335713] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10660.338180] LustreError: Skipped 1 previous similar message [10660.345763] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10660.358459] Lustre: Skipped 1 previous similar message [10663.394820] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10663.414735] Lustre: Skipped 2 previous similar messages [10666.620496] Lustre: server umount lustre-OST0000 complete [10672.096335] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10672.112056] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10672.130743] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10673.632269] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10676.868473] Lustre: server umount lustre-MDT0000 complete [10680.626325] LustreError: 298988:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787854828 with bad export cookie 8848336636982944307 [10680.634028] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10680.635097] LustreError: 298988:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10680.912957] Lustre: server umount lustre-MDT0001 complete [10689.494645] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10693.969670] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [10698.427848] Lustre: server umount MGS complete [10709.177135] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10709.641250] LustreError: 304780:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10709.657753] LustreError: 304780:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 3 previous similar messages [10709.667735] LustreError: 304793:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10709.738766] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10713.975594] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [10715.114373] LustreError: 304781:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10719.204253] LustreError: 304780:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10722.569517] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10723.000405] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10727.179272] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [10735.297243] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10741.733986] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10748.883029] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10749.293880] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10755.807813] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [10765.702899] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10795.490163] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10795.492057] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10795.494757] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10795.494765] Lustre: Skipped 2 previous similar messages [10795.515821] Lustre: Skipped 1 previous similar message [10798.546026] Lustre: server umount lustre-OST0000 complete [10804.710463] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10804.718952] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10804.728441] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10804.733412] Lustre: Skipped 1 previous similar message [10808.652171] Lustre: server umount lustre-MDT0000 complete [10810.852140] LustreError: 304781:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10810.870351] LustreError: 304781:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [10813.057378] LustreError: 304762:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787854961 with bad export cookie 8848336636982946162 [10813.063596] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10813.077913] LustreError: 304762:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10813.806068] Lustre: server umount lustre-MDT0001 complete [10826.614626] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [10829.680973] Key type lgssc unregistered [10829.920944] LNet: 308732:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10829.927045] LNetError: 308732:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10829.942739] LNet: Removed LNI 192.168.202.157@tcp [10830.842116] Key type .llcrypt unregistered [10830.845705] Key type ._llcrypt unregistered [10852.948526] Key type ._llcrypt registered [10852.952024] Key type .llcrypt registered [10853.133649] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_hostid [10870.163873] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [10871.270088] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10871.292542] alg: No test for adler32 (adler32-zlib) [10872.271387] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [10872.663145] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [10874.375361] Key type lgssc registered [10875.424835] Lustre: Echo OBD driver; http://www.lustre.org/ [10909.885434] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [10926.704239] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10926.775163] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10928.125868] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10928.179587] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10928.288060] Lustre: lustre-MDT0000: new disk, initializing [10928.358844] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10928.380623] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10933.309113] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [10943.450362] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10943.706413] Lustre: lustre-OST0000: new disk, initializing [10943.710972] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10943.718414] Lustre: Skipped 1 previous similar message [10943.723547] Lustre: 312785:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10943.784619] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10950.191757] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [10950.764213] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10950.773313] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10950.800917] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [11818.846939] Lustre: server umount lustre-MDT0000 complete [11823.958458] LustreError: 311970:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787855972 with bad export cookie 991087432527557305 [11823.976894] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11834.473509] Lustre: server umount lustre-OST0000 complete [11865.744732] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_hostid [11877.648257] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [11921.512029] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11921.947893] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11922.024086] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11922.173947] Lustre: lustre-MDT0000: new disk, initializing [11922.384641] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11922.427215] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11928.574617] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [11942.582902] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11942.724118] Lustre: 317129:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11942.803398] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11942.807262] Lustre: Skipped 1 previous similar message [11942.900621] Lustre: lustre-MDT0001: new disk, initializing [11943.010476] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11943.105750] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11943.115579] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11948.770371] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [11958.945818] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11964.968675] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11972.479290] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11972.768560] Lustre: lustre-OST0000: new disk, initializing [11972.777251] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11972.788798] Lustre: 319098:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11972.882392] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11979.720342] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [11982.999455] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11983.027209] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11983.128665] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11989.670970] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11994.735843] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11994.933387] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12000.131343] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12000.383646] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12003.821941] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12003.828814] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12003.849970] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12003.878261] Lustre: Skipped 1 previous similar message [12008.907995] Lustre: server umount lustre-OST0000 complete [12014.055144] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12014.060410] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12014.083411] Lustre: Skipped 2 previous similar messages [12014.109506] Lustre: Skipped 1 previous similar message [12019.176562] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12024.290518] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12024.292972] Lustre: Skipped 2 previous similar messages [12027.362956] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12027.509539] Lustre: server umount lustre-MDT0000 complete [12031.358994] LustreError: 317121:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787856179 with bad export cookie 991087432530498131 [12031.367836] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12031.370656] LustreError: 317121:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12031.754650] Lustre: server umount lustre-MDT0001 complete [12036.124493] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [12038.890720] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 14:43:05 (1787856185) [12047.616698] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12048.168159] LustreError: 321414:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12048.180151] LustreError: 321414:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 3 previous similar messages [12048.226418] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12053.484739] LustreError: 321415:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12054.707259] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [12058.606035] LustreError: 321414:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12063.728378] LustreError: 321415:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12064.845226] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12065.698124] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12071.857049] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [12079.623580] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12086.635669] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12094.562836] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12094.949659] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12101.870276] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [12111.060366] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12119.870513] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12120.168597] Lustre: lustre-OST0001: new disk, initializing [12120.173392] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [12120.181562] Lustre: 324250:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12120.359828] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12128.066211] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [12129.927809] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [12129.940164] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [12129.992842] LustreError: 324711:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12131.058185] LustreError: 324715:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12131.071275] LustreError: 324715:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [12132.139356] LustreError: 324715:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12132.160749] LustreError: 324715:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [12134.241776] LustreError: 324715:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12134.263402] LustreError: 324715:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [12138.337235] LustreError: 324711:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12138.343779] LustreError: 324711:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [12140.122775] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12140.367854] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12146.428848] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff999887cea000.ost_server_uuid 50 [12146.531663] LustreError: 324711:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12146.543495] LustreError: 324711:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [12148.462987] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff999887cea000.ost_server_uuid in FULL state after 0 sec [12156.048176] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff999887cea000.ost_server_uuid 50 [12157.973063] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff999887cea000.ost_server_uuid in IDLE state after 0 sec [12165.114322] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [12175.176475] Lustre: lustre-OST0001: Not available for connect from 192.168.202.57@tcp (stopping) [12175.339354] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12175.348328] LustreError: Skipped 1 previous similar message [12175.353056] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12180.454256] Lustre: server umount lustre-OST0001 complete [12181.502329] LustreError: 323237:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12181.518398] LustreError: 323237:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [12190.176694] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12190.200159] LustreError: Skipped 1 previous similar message [12190.209955] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12190.238569] Lustre: Skipped 1 previous similar message [12190.255180] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12190.277683] Lustre: Skipped 5 previous similar messages [12191.717297] LustreError: 323235:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12191.734731] LustreError: 323235:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 4 previous similar messages [12194.379434] Lustre: server umount lustre-OST0000 complete [12201.953819] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12201.970993] Lustre: Skipped 2 previous similar messages [12205.665656] Lustre: server umount lustre-MDT0000 complete [12210.339812] LustreError: 321396:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787856358 with bad export cookie 991087432530499013 [12210.358162] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12211.042494] Lustre: server umount lustre-MDT0001 complete [12224.397555] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [12227.656600] Key type lgssc unregistered [12227.984856] LNet: 326948:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12227.993468] LNetError: 326948:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12228.015921] LNet: Removed LNI 192.168.202.157@tcp [12229.081981] Key type .llcrypt unregistered [12229.084412] Key type ._llcrypt unregistered [12256.107885] Key type ._llcrypt registered [12256.114206] Key type .llcrypt registered [12256.343043] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [12257.692197] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12257.775454] alg: No test for adler32 (adler32-zlib) [12258.850766] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [12259.122158] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [12260.783204] Key type lgssc registered [12262.028900] Lustre: Echo OBD driver; http://www.lustre.org/ [12270.607470] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12270.641905] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12272.376693] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12277.133546] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [12282.848445] LustreError: 328368:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12282.882049] LustreError: 328368:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 2 previous similar messages [12286.159490] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12287.972724] LustreError: 328369:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12293.090510] LustreError: 328368:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12294.096784] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12294.652964] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12300.330363] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [12312.072904] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12312.440493] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12318.573029] LustreError: 330048:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12318.622963] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [12319.744481] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [12324.844089] LustreError: 330049:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12324.858097] LustreError: 330049:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 3 previous similar messages [12328.210051] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12335.082531] LustreError: 330398:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12335.155770] LustreError: 330398:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 3 previous similar messages [12338.270811] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12338.515140] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12343.800667] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [12345.335535] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [12356.489584] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12363.434727] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [12363.759399] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12471.785981] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12471.798411] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12471.800194] LustreError: Skipped 1 previous similar message [12471.809475] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12471.815297] Lustre: Skipped 1 previous similar message [12474.000843] Lustre: server umount lustre-OST0001 complete [12476.899419] LustreError: 330048:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12476.924414] LustreError: 330048:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [12482.020197] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12482.024035] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12482.035854] Lustre: Skipped 1 previous similar message [12482.039444] Lustre: Skipped 1 previous similar message [12487.139686] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12487.144390] Lustre: Skipped 1 previous similar message [12492.265371] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12492.275046] Lustre: Skipped 1 previous similar message [12494.303697] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12494.494785] Lustre: server umount lustre-OST0000 complete [12499.435556] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12499.457409] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12499.484937] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12502.501482] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12505.120157] Lustre: server umount lustre-MDT0000 complete [12509.569082] LustreError: 328349:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787856657 with bad export cookie 9932789864497002686 [12509.575849] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12509.585024] LustreError: 328349:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12510.033236] Lustre: server umount lustre-MDT0001 complete [12523.344993] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [12526.696636] Key type lgssc unregistered [12527.064725] LNet: 333317:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12527.073085] LNetError: 333317:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12527.091976] LNet: Removed LNI 192.168.202.157@tcp [12528.110153] Key type .llcrypt unregistered [12528.112514] Key type ._llcrypt unregistered [12581.077340] Key type ._llcrypt registered [12581.081410] Key type .llcrypt registered [12581.198329] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [12582.323750] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12582.399331] alg: No test for adler32 (adler32-zlib) [12583.591355] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [12583.945749] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [12585.713836] Key type lgssc registered [12587.134435] Lustre: Echo OBD driver; http://www.lustre.org/ [12598.146891] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12598.174850] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12599.803503] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12604.708204] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [12609.002264] LustreError: 335612:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12609.030749] LustreError: 335612:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 2 previous similar messages [12614.116507] LustreError: 335611:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12615.047556] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12615.503848] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12620.923318] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [12624.532814] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12632.337208] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12632.778696] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12639.993561] LustreError: 337102:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12640.198822] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [12644.066874] LustreError: 337104:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12644.085273] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12644.098222] LustreError: 337104:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [12649.448882] LustreError: 337103:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12649.471846] LustreError: 337103:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [12649.677680] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12649.895475] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12655.119381] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12656.821760] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [12667.322229] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12673.875866] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12687.340303] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12687.350449] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12687.367658] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12690.913828] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12690.937040] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12690.947414] Lustre: Skipped 2 previous similar messages [12693.005435] Lustre: server umount lustre-MDT0000 complete [12696.036815] LustreError: 336591:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12696.064178] LustreError: 336591:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 3 previous similar messages [12697.398492] LustreError: 335592:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787856845 with bad export cookie 2915039729343792739 [12697.413484] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12697.417132] LustreError: 335592:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12697.752409] Lustre: server umount lustre-MDT0001 complete [12712.500635] Lustre: server umount lustre-OST0000 complete [12726.932222] Lustre: server umount lustre-OST0001 complete [12746.617628] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [12757.764977] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12758.319439] LustreError: 340875:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12758.395626] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12762.839311] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [12773.125282] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12773.637820] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12777.775133] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [12780.623210] Lustre: 342015:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12787.278943] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12795.553218] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [12798.162658] LustreError: 342369:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12798.196355] LustreError: 342369:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 2 previous similar messages [12803.339050] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12806.656850] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12807.071505] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12807.080433] Lustre: Skipped 1 previous similar message [12812.294183] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12814.865305] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [12823.735458] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12828.619223] Lustre: 343888:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12835.301215] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12835.312280] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12835.325572] Lustre: Skipped 2 previous similar messages [12835.329403] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12835.332315] Lustre: Skipped 1 previous similar message [12837.878989] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12837.889867] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12837.915399] Lustre: Skipped 1 previous similar message [12840.189689] Lustre: server umount lustre-MDT0000 complete [12844.250615] LustreError: 340858:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787856992 with bad export cookie 2915039729343794440 [12844.252798] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12844.267713] LustreError: 340858:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12844.734253] Lustre: server umount lustre-MDT0001 complete [12859.375048] Lustre: server umount lustre-OST0000 complete [12873.670944] Lustre: server umount lustre-OST0001 complete [12878.530051] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12880.459142] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12883.037419] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 14:57:09 (1787857029) [12910.109899] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [12920.026771] loop0: detected capacity change from 0 to 1099511627776 [12920.656960] SGI XFS with ACLs, security attributes, quota, no debug enabled [12920.670996] *** VALIDATE xfs *** [12920.696074] XFS (loop0): Mounting V5 Filesystem [12920.737818] XFS (loop0): Ending clean mount [12921.985403] loop1: detected capacity change from 0 to 18691697672192 [13048.337833] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_hostid [13059.183635] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [13099.085170] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13099.435404] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13099.458325] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13099.561968] Lustre: lustre-MDT0000: new disk, initializing [13099.651633] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13099.665563] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13104.493718] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [13116.788555] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13116.917565] Lustre: 349043:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13116.986915] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13116.990722] Lustre: Skipped 1 previous similar message [13117.092403] Lustre: lustre-MDT0001: new disk, initializing [13117.315873] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13117.334120] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13122.898323] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [13133.458639] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13139.300513] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13146.920750] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13147.277509] Lustre: lustre-OST0000: new disk, initializing [13147.280372] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13147.286668] Lustre: 351025:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13148.534671] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13148.545080] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13148.699644] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13154.923944] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [13166.047107] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13171.782776] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [13172.216429] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [13178.887446] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [13179.193292] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [13184.482676] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [13184.492966] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13184.516503] Lustre: Skipped 1 previous similar message [13184.528068] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13184.543500] Lustre: Skipped 4 previous similar messages [13187.574607] Lustre: server umount lustre-OST0000 complete [13194.209378] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13194.214888] LustreError: Skipped 1 previous similar message [13194.220575] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13194.228635] Lustre: Skipped 1 previous similar message [13194.231772] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13198.917595] Lustre: server umount lustre-MDT0000 complete [13199.840804] LustreError: 349052:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13199.852227] LustreError: 349052:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 7 previous similar messages [13203.227970] LustreError: 349851:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787857351 with bad export cookie 2915039729343795994 [13203.251257] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13203.858835] Lustre: server umount lustre-MDT0001 complete [13213.708307] XFS (loop0): Unmounting Filesystem [13219.625033] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 15:02:46 (1787857366) [13227.059731] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13227.825713] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13227.828669] Lustre: Skipped 2 previous similar messages [13232.932349] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [13242.467953] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13248.408632] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [13258.885774] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13265.593283] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13275.581572] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13284.801400] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [13293.969356] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13308.393062] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [13308.395758] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13308.409061] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13308.409070] Lustre: Skipped 1 previous similar message [13308.442886] Lustre: Skipped 2 previous similar messages [13313.896885] Lustre: server umount lustre-OST0000 complete [13319.653438] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13319.666902] LustreError: Skipped 1 previous similar message [13325.328920] Lustre: server umount lustre-MDT0000 complete [13328.879236] LustreError: 353584:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13328.909029] LustreError: 353584:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 5 previous similar messages [13330.387216] LustreError: 353570:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787857478 with bad export cookie 2915039729343796869 [13330.393926] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13330.408995] LustreError: 353570:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [13331.105800] Lustre: server umount lustre-MDT0001 complete [13341.751722] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [13344.442624] Key type lgssc unregistered [13344.704521] LNet: 357223:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13344.711611] LNetError: 357223:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13344.733278] LNet: Removed LNI 192.168.202.157@tcp [13345.541214] Key type .llcrypt unregistered [13345.543696] Key type ._llcrypt unregistered [13373.155851] Key type ._llcrypt registered [13373.159267] Key type .llcrypt registered [13373.277591] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [13374.420460] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13374.550708] alg: No test for adler32 (adler32-zlib) [13375.683804] Lustre: Lustre: Build Version: 2.17.57_89_g2dd35b7 [13375.935355] LNet: Added LNI 192.168.202.157@tcp [8/256/0/180] [13377.615219] Key type lgssc registered [13378.840971] Lustre: Echo OBD driver; http://www.lustre.org/ [13385.717301] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13385.755879] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13387.207649] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13392.317483] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [13397.484233] LustreError: 358792:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13397.502726] LustreError: 358792:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 2 previous similar messages [13401.318222] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13401.880665] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13407.057450] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [13414.704735] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13420.425796] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13428.468482] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13428.840143] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13436.961904] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [13446.892953] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13456.869995] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13456.876712] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13456.879813] Lustre: Skipped 1 previous similar message [13459.733825] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.57@tcp (stopping) [13461.519807] Lustre: server umount lustre-MDT0001 complete [13461.987137] LustreError: 358793:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13462.000036] LustreError: 358793:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [13464.831590] LustreError: 358789:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.57@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13467.109496] LustreError: 358792:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13467.118085] LustreError: 358792:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [13471.945296] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13472.017185] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13472.021954] Lustre: Found index 1 for lustre-MDT0001, updating log [13472.055987] Lustre: 358779:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13472.306784] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13477.247578] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [13477.354506] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13482.530765] LustreError: 362396:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13482.536112] LustreError: 362396:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13482.552867] LustreError: 358784:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13482.636186] LustreError: 362399:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.157@tcp: cfg command failed: rc = -17 [13482.645905] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [13482.645905] [13782.312073] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.57@tcp (stopping) [13784.550451] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13784.552375] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13784.559149] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13784.565718] Lustre: Skipped 2 previous similar messages [13784.574817] Lustre: Skipped 1 previous similar message [13788.495327] Lustre: server umount lustre-MDT0001 complete [13789.690780] LustreError: 358792:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13789.737329] LustreError: 358792:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 2 previous similar messages [13799.927502] LustreError: 358793:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13799.938257] LustreError: 358793:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 4 previous similar messages [13800.624219] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13800.696221] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13800.699688] Lustre: Found index 1 for lustre-MDT0001, updating log [13800.727277] Lustre: 358780:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13801.290681] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13806.585230] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13808.044660] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [13810.237252] LustreError: 370456:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13810.243982] LustreError: 370456:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13810.254194] LustreError: 358784:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13810.272418] LustreError: 358784:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13810.492367] LustreError: 370470:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13810.526566] LustreError: 370470:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13810.536228] LustreError: 370470:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.157@tcp: cfg command failed: rc = -17 [13810.571201] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.157@tcp [13810.571201] [14118.882909] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14118.891559] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [14118.906340] Lustre: Skipped 1 previous similar message [14118.921896] Lustre: Skipped 2 previous similar messages [14122.000879] Lustre: server umount lustre-MDT0001 complete [14124.002638] LustreError: 358789:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14124.018329] LustreError: 358789:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 1 previous similar message [14133.321289] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14133.432887] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [14133.444418] Lustre: Found index 1 for lustre-MDT0001, updating log [14133.542345] Lustre: 358780:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14133.985915] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [14139.385288] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [14139.468224] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [14141.453058] LustreError: 378363:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [14141.468915] LustreError: 378363:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [14141.477354] LustreError: 358784:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [14141.489991] LustreError: 358784:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [14141.600679] LustreError: 378369:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [14141.615920] LustreError: 378369:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.157@tcp: cfg command failed: rc = -114 [14141.623976] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [14141.623976] [14467.042548] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14467.051395] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14467.058875] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14467.059711] Lustre: Skipped 1 previous similar message [14467.120269] Lustre: Skipped 2 previous similar messages [14468.998563] Lustre: server umount lustre-MDT0000 complete [14472.163160] LustreError: 362244:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14472.191306] LustreError: 362244:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 7 previous similar messages [14473.623406] LustreError: 361979:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787858621 with bad export cookie 4013137276982458502 [14473.636615] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14473.641460] LustreError: 361979:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14477.282398] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14480.128378] Lustre: server umount lustre-MDT0001 complete [14485.472716] Lustre: server umount lustre-OST0000 complete [14498.127947] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_hostid [14510.269527] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [14554.544222] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 15:25:01 (1787858701) [14562.537829] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14562.752405] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14562.770298] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14562.839453] Lustre: lustre-MDT0000: new disk, initializing [14562.908326] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14562.926464] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14568.401559] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [14581.083312] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14581.212666] Lustre: 388327:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14581.265699] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14581.268362] Lustre: Skipped 1 previous similar message [14581.334125] Lustre: lustre-MDT0001: new disk, initializing [14581.419727] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14581.463741] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14581.485770] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14586.952722] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [14600.528403] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14609.762254] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14619.578701] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14620.080162] Lustre: lustre-OST0000: new disk, initializing [14620.083290] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14620.089585] Lustre: 390300:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14620.213922] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14627.467628] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14627.487064] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14627.640082] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14628.293817] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_default_debug -1 all [14641.534429] Lustre: DEBUG MARKER: oleg257-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14653.409740] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14653.425290] Lustre: Skipped 1 previous similar message [14653.426221] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14653.441057] Lustre: Skipped 5 previous similar messages [14657.618822] Lustre: server umount lustre-MDT0000 complete [14658.534099] LustreError: 388340:0:(ldlm_lib.c:1212:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14658.559296] LustreError: 388340:0:(ldlm_lib.c:1212:target_handle_connect()) Skipped 4 previous similar messages [14662.089685] LustreError: 390299:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787858810 with bad export cookie 4013137276982460448 [14662.090942] LustreError: MGC192.168.202.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14662.102432] LustreError: 390299:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14662.445202] Lustre: server umount lustre-MDT0001 complete [14678.164805] Lustre: server umount lustre-OST0000 complete [14714.366784] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing set_hostid [14729.759461] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing load_modules_local [14769.336950] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 14481 sec ========= 15:28:35 (1787858915) [14771.651778] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 15:28:37 (1787858917) === [14776.058888] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 15:28:42 (1787858922) === [14826.404440] Lustre: DEBUG MARKER: oleg257-server.virtnet: executing unload_modules_local [14829.709452] Key type lgssc unregistered [14830.078244] LNet: 396696:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14830.087320] LNetError: 396696:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14830.103633] LNet: Removed LNI 192.168.202.157@tcp [14830.978692] Key type .llcrypt unregistered [14830.982574] Key type ._llcrypt unregistered