[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-8.fc42 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 572946177 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2439 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE22D5 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 002295 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2349 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE23D9 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2411 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe22d5-0xbffe2348] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22d4] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2349-0xbffe23d8] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe23d9-0xbffe2410] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2411-0xbffe2438] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001016] APIC: Switch to symmetric I/O mode setup [ 0.003087] x2apic enabled [ 0.004012] Switched APIC routing to physical x2apic. [ 0.005020] kvm-guest: setup PV IPIs [ 0.008675] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009038] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010028] pid_max: default: 32768 minimum: 301 [ 0.011177] LSM: Security Framework initializing [ 0.012091] Yama: becoming mindful. [ 0.013105] SELinux: Initializing. [ 0.015053] *** VALIDATE selinux *** [ 0.025099] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.030599] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.032056] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.033134] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.034158] *** VALIDATE tmpfs *** [ 0.035540] *** VALIDATE proc *** [ 0.036321] *** VALIDATE cgroup *** [ 0.037021] *** VALIDATE cgroup2 *** [ 0.038296] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.039200] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.040015] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.041043] Spectre V2 : User space: Vulnerable [ 0.042015] Speculative Store Bypass: Vulnerable [ 0.045483] debug: unmapping init [mem 0xffffffffb8a59000-0xffffffffb8a60fff] [ 0.048000] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.048919] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.049034] ... version: 2 [ 0.050020] ... bit width: 48 [ 0.051014] ... generic registers: 4 [ 0.052020] ... value mask: 0000ffffffffffff [ 0.053022] ... max period: 00007fffffffffff [ 0.054020] ... fixed-purpose events: 3 [ 0.055020] ... event mask: 000000070000000f [ 0.056383] rcu: Hierarchical SRCU implementation. [ 0.059114] smp: Bringing up secondary CPUs ... [ 0.060826] x86: Booting SMP configuration: [ 0.061054] .... node #0, CPUs: #1 #2 #3 [ 0.067033] smp: Brought up 1 node, 4 CPUs [ 0.069024] smpboot: Max logical packages: 1 [ 0.070031] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.157473] node 0 deferred pages initialised in 84ms [ 0.164021] devtmpfs: initialized [ 0.166535] x86/mm: Memory block size: 128MB [ 0.170010] gcov: version magic: 0x41383552 [ 0.173330] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.174104] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.175441] pinctrl core: initialized pinctrl subsystem [ 0.176389] [ 0.177011] ************************************************************* [ 0.178019] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.179023] ** ** [ 0.180027] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.181025] ** ** [ 0.182025] ** This means that this kernel is built to expose internal ** [ 0.183021] ** IOMMU data structures, which may compromise security on ** [ 0.184028] ** your system. ** [ 0.185019] ** ** [ 0.186027] ** If you see this message and you are not debugging the ** [ 0.187020] ** kernel, report this immediately to your vendor! ** [ 0.188356] ** ** [ 0.189022] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.190027] ************************************************************* [ 0.192400] NET: Registered protocol family 16 [ 0.193691] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.194096] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.195096] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.197036] cpuidle: using governor menu [ 0.200700] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.205308] PCI: Using configuration type 1 for base access [ 0.208159] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.220232] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.223166] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.229188] cryptd: max_cpu_qlen set to 1000 [ 0.232270] ACPI: Added _OSI(Module Device) [ 0.233015] ACPI: Added _OSI(Processor Device) [ 0.234016] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.235019] ACPI: Added _OSI(Processor Aggregator Device) [ 0.241221] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.245026] ACPI: Interpreter enabled [ 0.246075] ACPI: PM: (supports S0 S3 S4 S5) [ 0.247020] ACPI: Using IOAPIC for interrupt routing [ 0.248182] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.249594] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.264276] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.265066] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.266031] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.267107] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.270185] acpiphp: Slot [2] registered [ 0.271217] acpiphp: Slot [5] registered [ 0.272355] acpiphp: Slot [6] registered [ 0.273421] acpiphp: Slot [7] registered [ 0.274337] acpiphp: Slot [8] registered [ 0.275388] acpiphp: Slot [9] registered [ 0.276468] acpiphp: Slot [10] registered [ 0.277341] acpiphp: Slot [3] registered [ 0.278434] acpiphp: Slot [4] registered [ 0.279267] acpiphp: Slot [11] registered [ 0.280321] acpiphp: Slot [12] registered [ 0.281160] acpiphp: Slot [13] registered [ 0.282386] acpiphp: Slot [14] registered [ 0.283168] acpiphp: Slot [15] registered [ 0.284156] acpiphp: Slot [16] registered [ 0.285419] acpiphp: Slot [17] registered [ 0.286337] acpiphp: Slot [18] registered [ 0.287198] acpiphp: Slot [19] registered [ 0.288324] acpiphp: Slot [20] registered [ 0.289630] acpiphp: Slot [21] registered [ 0.290178] acpiphp: Slot [22] registered [ 0.291810] acpiphp: Slot [23] registered [ 0.292329] acpiphp: Slot [24] registered [ 0.293508] acpiphp: Slot [25] registered [ 0.294334] acpiphp: Slot [26] registered [ 0.295203] acpiphp: Slot [27] registered [ 0.296902] acpiphp: Slot [28] registered [ 0.297211] acpiphp: Slot [29] registered [ 0.298137] acpiphp: Slot [30] registered [ 0.299242] acpiphp: Slot [31] registered [ 0.300081] PCI host bridge to bus 0000:00 [ 0.301025] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.302127] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.303030] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.304097] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.305029] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.306031] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.307193] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.309307] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.311482] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.317677] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.320051] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.321022] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.322022] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.323020] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.324725] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.325820] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.326045] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.328027] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.330750] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.338020] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.341022] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.344000] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.355019] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.364022] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.406035] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.418847] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.428021] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.442024] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.460025] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.470000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.483024] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.494032] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.495000] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.495000] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.495000] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.522025] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.698026] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.736416] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 1.603031] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 1.630019] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 1.658023] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 1.674951] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 1.689020] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 1.699021] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 1.727021] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 1.742056] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 1.744909] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 1.746455] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 1.748357] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 1.750303] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 1.755482] iommu: Default domain type: Passthrough [ 1.757066] SCSI subsystem initialized [ 1.758173] ACPI: bus type USB registered [ 1.760236] usbcore: registered new interface driver usbfs [ 1.761122] usbcore: registered new interface driver hub [ 1.763324] usbcore: registered new device driver usb [ 1.764215] pps_core: LinuxPPS API ver. 1 registered [ 1.765000] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 1.765000] PTP clock support registered [ 1.770070] EDAC MC: Ver: 3.0.0 [ 1.774088] PCI: Using ACPI for IRQ routing [ 1.778017] NetLabel: Initializing [ 1.783018] NetLabel: domain hash size = 128 [ 1.789094] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 1.871118] NetLabel: unlabeled traffic allowed by default [ 1.875569] vgaarb: loaded [ 1.879008] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 1.882019] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 1.890603] clocksource: Switched to clocksource kvm-clock [ 2.176155] VFS: Disk quotas dquot_6.6.0 [ 2.181472] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 2.189209] *** VALIDATE ramfs *** [ 2.191054] *** VALIDATE hugetlbfs *** [ 2.193367] pnp: PnP ACPI init [ 2.197739] pnp: PnP ACPI: found 6 devices [ 2.218056] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 2.222582] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 2.225685] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 2.228391] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 2.231406] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 2.234750] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 2.237932] NET: Registered protocol family 2 [ 2.240711] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 2.247123] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 2.252065] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 2.288214] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 2.292264] TCP: Hash tables configured (established 65536 bind 65536) [ 2.295730] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 2.299191] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 2.302284] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 2.305564] NET: Registered protocol family 1 [ 2.308573] RPC: Registered named UNIX socket transport module. [ 2.314604] RPC: Registered udp transport module. [ 2.316601] RPC: Registered tcp transport module. [ 2.318421] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 2.320974] NET: Registered protocol family 44 [ 2.322344] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 2.324752] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 2.327166] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 2.329852] PCI: CLS 0 bytes, default 64 [ 2.332260] Unpacking initramfs... [ 5.724682] debug: unmapping init [mem 0xffff9fc0bcc54000-0xffff9fc0bffbffff] [ 5.729221] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 5.731239] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 5.733827] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 6.526776] Initialise system trusted keyrings [ 6.529069] Key type blacklist registered [ 6.531399] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 6.544985] zbud: loaded [ 6.549046] *** VALIDATE nfs *** [ 6.550620] *** VALIDATE nfs4 *** [ 6.552985] pstore: using deflate compression [ 6.558753] Platform Keyring initialized [ 6.722861] NET: Registered protocol family 38 [ 6.725287] Key type asymmetric registered [ 6.727133] Asymmetric key parser 'x509' registered [ 6.729191] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 6.734950] io scheduler mq-deadline registered [ 6.737115] io scheduler kyber registered [ 6.738715] io scheduler bfq registered [ 6.740364] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 6.743368] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 6.746738] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 6.749758] ACPI: Power Button [PWRF] [ 6.755639] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 6.763203] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 6.780720] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 6.789531] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 6.808462] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 6.838667] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 6.868947] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 6.873351] Non-volatile memory driver v1.3 [ 6.875271] Linux agpgart interface v0.103 [ 6.908214] virtio_blk virtio1: [vda] 134032 512-byte logical blocks (68.6 MB/65.4 MiB) [ 6.912476] vda: detected capacity change from 0 to 68624384 [ 7.101778] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 7.105838] vdb: detected capacity change from 0 to 1073741824 [ 7.213206] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 7.217201] vdc: detected capacity change from 0 to 2621440000 [ 7.273175] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 7.278261] vdd: detected capacity change from 0 to 2621440000 [ 7.344501] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 7.349894] vde: detected capacity change from 0 to 4294967296 [ 7.394351] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 7.398995] vdf: detected capacity change from 0 to 4294967296 [ 7.409645] libphy: Fixed MDIO Bus: probed [ 7.423799] usbcore: registered new interface driver usbserial_generic [ 7.426417] usbserial: USB Serial support registered for generic [ 7.428749] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 7.436270] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 7.438207] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 7.441575] mousedev: PS/2 mouse device common for all mice [ 7.446356] rtc_cmos 00:05: RTC can wake from S4 [ 7.455856] rtc_cmos 00:05: registered as rtc0 [ 7.460174] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 7.460517] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 7.467565] intel_pstate: CPU model not supported [ 7.476166] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 7.478544] hid: raw HID events driver (C) Jiri Kosina [ 7.485164] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 7.486290] usbcore: registered new interface driver usbhid [ 7.486294] usbhid: USB HID core driver [ 7.486510] drop_monitor: Initializing network drop monitor service [ 7.486646] Initializing XFRM netlink socket [ 7.487048] NET: Registered protocol family 10 [ 7.488390] Segment Routing with IPv6 [ 7.488550] NET: Registered protocol family 17 [ 7.488725] mpls_gso: MPLS GSO support [ 7.515371] RAS: Correctable Errors collector initialized. [ 7.518027] AVX version of gcm_enc/dec engaged. [ 7.519897] AES CTR mode by8 optimization enabled [ 7.607703] sched_clock: Marking stable (7607626111, 0)->(9902426175, -2294800064) [ 7.614290] registered taskstats version 1 [ 7.617092] Loading compiled-in X.509 certificates [ 7.618666] zswap: loaded using pool lzo/zbud [ 7.644583] Key type big_key registered [ 7.659601] Key type encrypted registered [ 7.661554] ima: No TPM chip found, activating TPM-bypass! [ 7.663701] ima: Allocated hash algorithm: sha1 [ 7.665120] ima: No architecture policies found [ 7.666878] evm: Initialising EVM extended attributes: [ 7.668995] evm: security.selinux [ 7.670499] evm: security.ima [ 7.671678] evm: security.capability [ 7.673217] evm: HMAC attrs: 0x1 [ 7.675844] rtc_cmos 00:05: setting system clock to 2026-01-22 09:10:18 UTC (1769073018) [ 7.683996] debug: unmapping init [mem 0xffffffffb9a03000-0xffffffffb9bfffff] [ 7.687519] debug: unmapping init [mem 0xffffffffb8782000-0xffffffffb8a58fff] [ 7.692203] Write protecting the kernel read-only data: 28672k [ 7.695927] debug: unmapping init [mem 0xffffffffb6e03000-0xffffffffb6ffffff] [ 7.699179] debug: unmapping init [mem 0xffffffffb7714000-0xffffffffb77fffff] [ 7.741846] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 7.751523] systemd[1]: Detected virtualization kvm. [ 7.753774] systemd[1]: Detected architecture x86-64. [ 7.756189] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 7.786718] systemd[1]: No hostname configured. [ 7.788565] systemd[1]: Set hostname to . [ 7.790750] random: systemd: uninitialized urandom read (16 bytes read) [ 7.793274] systemd[1]: Initializing machine ID from random generator. [ 7.884775] random: ln: uninitialized urandom read (6 bytes read) [ 8.089278] random: systemd: uninitialized urandom read (16 bytes read) [ 8.092170] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 8.104585] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 8.114304] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Slices. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... Starting Journal Service... [ OK ] Started Memstrack Anylazing Service. Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... [ OK ] Reached target Timers. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Initrd Root Device. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 9.447562] device-mapper: uevent: version 1.0.3 [ 9.451295] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target B[ 11.676213] virtio_net virtio0 ens2: renamed from eth0 asic System. [ 11.714925] random: fast init done [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 13.380545] scsi host0: ata_piix [ 13.441674] scsi host1: ata_piix [ 13.444986] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 13.456515] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 18.335932] random: crng init done [ 18.340316] random: 7 urandom warning(s) missed due to ratelimiting [ 19.335610] dracut-initqueue[585]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 21.349848] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 23.388634] printk: systemd: 26 output lines suppressed due to ratelimiting [ 23.951777] SELinux: Disabled at runtime. [ 24.041494] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 24.053714] systemd[1]: Detected virtualization kvm. [ 24.058996] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 25.324595] systemd[1]: initrd-switch-root.service: Succeeded. [ 25.331397] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 25.343477] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 25.351033] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 25.356155] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 25.366234] systemd[1]: Starting Journal Service... Starting Journal Service... [ 25.383731] systemd[1]: Created slice User and Session Slice. [ OK ] Created slice User and Session Slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on udev Kernel Socket. Mounting POSIX Message Queue File System... [ OK ] Listening on udev Control Socket. [ OK ] Listening on initctl Compatibility Named Pipe. Mounting Kernel Debug File System... [ OK ] Reached target rpc_pipefs.target. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-getty.slice. [ OK ] Reached target Slices. [ OK ] Started Forward Password Requests to Wall Directory Watch. Starting Apply Kernel Variables... Activating swap /dev/disk/by-label/SWAP... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ 25.667641] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Mounting Huge Pages File System... Starting udev Coldplug all Devices... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Local Encrypted Volumes. Starting Remount Root and Kernel File Systems... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Journal Service. [ OK ] Started Apply Kernel Variables. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started udev Coldplug all Devices. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 27.094629] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 28.468127] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 28.646230] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 32.247393] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 32.757631] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (7s / no limit) [** ] A start job is running for Configur…-only root support (8s / no limit) [*** ] A start job is running for Configur…-only root support (9s / no limit)[ 35.628942] Key type dns_resolver registered [ *** ] A start job is running for Configur…only root support (10s / no limit)[ 36.055604] NFS: Registering the id_resolver key type [ 36.057102] Key type id_resolver registered [ 36.058188] Key type id_legacy registered [ *** ] A start job is running for Configur…only root support (10s / no limit) [ ***] A start job is running for Configur…only root support (11s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. Starting Restore /run/initramfs on shutdown... Starting Network Manager... [ OK ] Started irqbalance daemon. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Reached target sshd-keygen.target. Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started OpenSSH server daemon. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg252-server login: [ 90.361901] libcfs: loading out-of-tree module taints kernel. [ 90.401519] Key type ._llcrypt registered [ 90.403146] Key type .llcrypt registered [ 90.482486] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_hostid [ 104.464449] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 105.484390] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 105.492323] alg: No test for adler32 (adler32-zlib) [ 106.576395] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 107.150376] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 108.840486] Key type lgssc registered [ 109.878380] Lustre: Echo OBD driver; http://www.lustre.org/ [ 122.982190] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 124.295250] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 130.578109] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 135.920035] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 141.498322] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 153.384977] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 163.977944] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 164.028517] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 164.049734] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 165.251036] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 165.289503] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 165.362175] Lustre: lustre-MDT0000: new disk, initializing [ 165.439846] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 165.458839] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 168.921729] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 179.785853] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 179.894867] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 179.991589] Lustre: 6515:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 180.019570] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 180.022953] Lustre: Skipped 1 previous similar message [ 180.117723] Lustre: lustre-MDT0001: new disk, initializing [ 180.167436] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 180.187695] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 180.197267] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 183.634657] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 187.960136] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 195.818769] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 195.877181] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 196.079761] Lustre: lustre-OST0000: new disk, initializing [ 196.087074] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 196.137384] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 200.626985] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 202.780919] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 202.788121] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 202.846333] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 212.939080] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 213.019990] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 213.142883] Lustre: lustre-OST0001: new disk, initializing [ 213.159964] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 213.250332] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 218.780586] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 221.716191] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 221.726916] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 221.755271] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 229.494651] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 239.916299] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 249.895654] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing check_logdir /tmp/testlogs/ [ 253.365904] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing yml_node [ 257.144946] Lustre: DEBUG MARKER: Client: 2.17.50.4 [ 259.415442] Lustre: DEBUG MARKER: MDS: 2.17.50.4 [ 261.920984] Lustre: DEBUG MARKER: OSS: 2.17.50.4 [ 263.328059] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Jan 22 04:14:32 EST 2026 [ 278.687845] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 280.277856] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 297.953778] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 297.994936] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 298.062919] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 303.586188] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 303.590748] Lustre: Skipped 7 previous similar messages [ 304.916486] LustreError: 12220:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 305.653685] Lustre: server umount lustre-MDT0000 complete [ 310.740450] LustreError: 8435:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769073321 with bad export cookie 10223810400771011164 [ 310.741787] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 310.750700] LustreError: 8435:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 313.825496] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 313.826095] LustreError: 12119:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 313.834038] Lustre: Skipped 3 previous similar messages [ 313.834786] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 313.848200] LustreError: 12119:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 315.878509] LustreError: 6523:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 315.881288] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 315.930717] LustreError: 6523:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 315.977634] Lustre: Skipped 2 previous similar messages [ 321.003393] LustreError: 12119:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 321.007228] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 321.054210] Lustre: Skipped 1 previous similar message [ 325.090199] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 325.495472] LustreError: 12421:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 325.533169] LustreError: 12421:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 325.773865] Lustre: server umount lustre-MDT0001 complete [ 326.175335] hrtimer: interrupt took 8664578 ns [ 333.862130] LustreError: 12625:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 333.865877] LustreError: 12625:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 333.896977] Lustre: server umount lustre-OST0000 complete [ 342.604972] LustreError: 12827:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 342.642992] LustreError: 12827:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 343.207149] Lustre: server umount lustre-OST0001 complete [ 357.493694] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_hostid [ 370.691573] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 384.395596] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 392.819091] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 400.986465] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 409.171806] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 419.193147] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 419.331365] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 419.964214] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 420.061455] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 420.224195] Lustre: lustre-MDT0000: new disk, initializing [ 420.398737] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 420.448135] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 426.250490] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 439.422678] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 439.554974] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 439.640035] Lustre: 15016:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 439.744912] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 439.749708] Lustre: Skipped 1 previous similar message [ 439.918957] Lustre: lustre-MDT0001: new disk, initializing [ 440.082626] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 440.155953] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 440.161620] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 446.896748] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 457.505827] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 462.841588] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 473.447902] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 473.552250] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 473.903350] Lustre: lustre-OST0000: new disk, initializing [ 473.907749] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 474.055808] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 475.027519] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 475.040839] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 475.281670] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 482.374519] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 492.196809] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 498.466502] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 498.752556] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 505.120751] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 505.405227] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 510.949083] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 510.954751] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 510.955690] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 510.980658] Lustre: Skipped 2 previous similar messages [ 522.212028] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 522.261247] LustreError: 17962:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 522.265900] LustreError: 17962:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 522.548897] Lustre: server umount lustre-OST0000 complete [ 531.434849] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 531.444690] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 531.465284] Lustre: Skipped 1 previous similar message [ 531.481785] Lustre: Skipped 6 previous similar messages [ 534.284882] LustreError: 18165:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 534.304328] LustreError: 18165:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 534.690711] Lustre: server umount lustre-MDT0000 complete [ 536.549800] LustreError: 15025:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 536.577907] LustreError: 15025:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 540.225649] LustreError: 15010:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769073551 with bad export cookie 10223810400771012697 [ 540.227869] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 540.239884] LustreError: 15010:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 541.003834] Lustre: server umount lustre-MDT0001 complete [ 547.399827] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 04:19:15 (1769073555) [ 556.571268] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 557.404810] LustreError: 18893:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 557.553392] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 562.827724] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 567.780102] LustreError: 18892:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 567.806055] LustreError: 18892:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 574.720411] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 581.158710] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 588.207178] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 593.342526] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 602.117029] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 610.265380] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 617.134197] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 624.668258] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 636.388931] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 636.400728] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 636.417221] Lustre: Skipped 2 previous similar messages [ 636.422175] Lustre: Skipped 1 previous similar message [ 639.288851] LustreError: 21197:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 639.299875] LustreError: 21197:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 639.654168] Lustre: server umount lustre-MDT0000 complete [ 641.518385] LustreError: 18892:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 641.539447] LustreError: 18892:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 644.779017] LustreError: 18874:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769073655 with bad export cookie 10223810400771013579 [ 644.783230] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 644.794935] LustreError: 18874:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 645.240489] Lustre: server umount lustre-MDT0001 complete [ 661.213344] Lustre: server umount lustre-OST0000 complete [ 692.996927] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 705.650307] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 706.582961] LustreError: 22956:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 706.675479] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 706.683538] Lustre: Skipped 2 previous similar messages [ 711.036934] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 722.094639] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 729.587346] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 734.049248] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 744.140814] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 752.995930] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 764.808721] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 764.966443] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 765.285698] Lustre: lustre-OST0001: new disk, initializing [ 765.297577] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 767.271913] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 767.285778] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 767.408867] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 771.945872] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 784.071104] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 788.827729] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 809.956519] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 809.962552] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 809.986368] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 815.888530] LustreError: 26442:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 815.898235] LustreError: 26442:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 816.107583] Lustre: server umount lustre-MDT0000 complete [ 818.688823] LustreError: 22956:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 818.714144] LustreError: 22956:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 821.230382] LustreError: 25229:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769073832 with bad export cookie 10223810400771014426 [ 821.251973] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 822.085710] Lustre: server umount lustre-MDT0001 complete [ 838.135849] Lustre: server umount lustre-OST0000 complete [ 839.973198] Lustre: 3657:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1769073834/real 1769073834] req@ffff9fc004553100 x1855007612989440/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1769073850 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 840.016958] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 840.046386] Lustre: Skipped 3 previous similar messages [ 842.208323] Lustre: 3659:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1769073837/real 1769073837] req@ffff9fc12ba18700 x1855007612989696/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1769073853 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 842.580457] Lustre: server umount lustre-OST0001 complete [ 864.963142] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 879.246266] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 880.236985] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 880.243275] Lustre: Skipped 3 previous similar messages [ 885.708564] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 898.034451] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 905.148824] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 909.219174] Lustre: 29338:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 918.467775] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 927.413237] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 938.022366] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 945.812863] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 955.863295] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 960.934907] Lustre: 31196:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 974.312144] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 974.316996] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 974.349908] Lustre: Skipped 2 previous similar messages [ 974.367815] Lustre: Skipped 7 previous similar messages [ 980.149385] LustreError: 31398:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 980.167624] LustreError: 31398:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 980.716332] Lustre: server umount lustre-MDT0000 complete [ 984.551058] LustreError: 28221:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 984.569628] LustreError: 28221:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 14 previous similar messages [ 985.816712] LustreError: 28201:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769073996 with bad export cookie 10223810400771015721 [ 985.818190] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 985.827649] LustreError: 28201:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 986.323915] Lustre: server umount lustre-MDT0001 complete [ 1001.263720] Lustre: server umount lustre-OST0000 complete [ 1006.053277] Lustre: 3659:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1769074000/real 1769074000] req@ffff9fc13d756d80 x1855007613049856/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1769074016 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1006.644009] Lustre: server umount lustre-OST0001 complete [ 1018.551608] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 04:27:06 (1769074026) [ 1055.059941] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 1069.131322] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1075.599842] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 1086.876884] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1091.842749] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 1094.940994] Lustre: 35105:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1104.562665] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1113.208818] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 1124.818181] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1133.810734] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 1144.100251] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1148.974523] Lustre: 36963:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1161.187294] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1161.231056] Lustre: Skipped 4 previous similar messages [ 1166.810719] LustreError: 37117:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 1166.818562] LustreError: 37117:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 1166.991945] Lustre: server umount lustre-MDT0000 complete [ 1171.640680] LustreError: 33966:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769074182 with bad export cookie 10223810400771016827 [ 1171.653203] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1171.657157] LustreError: 33966:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1172.451924] Lustre: server umount lustre-MDT0001 complete [ 1187.642848] Lustre: server umount lustre-OST0000 complete [ 1192.928845] Lustre: 3657:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1769074187/real 1769074187] req@ffff9fc004a12d80 x1855007613108864/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1769074203 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1193.399384] Lustre: server umount lustre-OST0001 complete [ 1204.429312] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 04:30:12 (1769074212) [ 1223.729412] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 1237.468199] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1238.077721] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1238.080821] Lustre: Skipped 7 previous similar messages [ 1243.128324] LustreError: 39092:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1243.188220] LustreError: 39092:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 15 previous similar messages [ 1244.348674] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 1252.608901] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1258.618132] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 1262.309715] Lustre: 40208:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1270.849119] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1278.658848] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 1289.948152] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1297.950308] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 1307.342264] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1312.146770] Lustre: 42065:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1318.390676] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1318.394647] Lustre: Skipped 1 previous similar message [ 1324.514849] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1324.527794] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1324.550024] Lustre: Skipped 2 previous similar messages [ 1324.557913] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1324.569368] Lustre: Skipped 12 previous similar messages [ 1325.057257] Lustre: server umount lustre-MDT0000 complete [ 1329.985535] LustreError: 42067:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769074340 with bad export cookie 10223810400771017933 [ 1330.007849] LustreError: 42067:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1330.014182] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1336.662389] Lustre: server umount lustre-MDT0001 complete [ 1347.681783] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1354.208881] LustreError: 3656:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9fc137a18700 x1855007613170176/t0(0) o250->MGC192.168.202.152@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1359.599658] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 1365.124722] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1369.006890] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1374.721803] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1374.735493] Lustre: Skipped 1 previous similar message [ 1375.303475] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 1382.337094] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1384.022992] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1388.501760] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1391.037713] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1400.308467] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1405.091517] Lustre: server umount lustre-MDT0000 complete [ 1409.149070] LustreError: 39071:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769074419 with bad export cookie 10223810400771019095 [ 1409.157268] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1409.160664] LustreError: 39071:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1409.539910] Lustre: server umount lustre-MDT0001 complete [ 1424.665290] LustreError: 44871:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 1424.668029] LustreError: 44871:0:(obd_class.h:479:obd_check_dev()) Skipped 57 previous similar messages [ 1424.715238] Lustre: server umount lustre-OST0000 complete [ 1426.657361] Lustre: 3660:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1769074421/real 1769074421] req@ffff9fc12a0fdf80 x1855007613204352/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1769074437 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1428.810846] Lustre: server umount lustre-OST0001 complete [ 1440.884730] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 04:34:09 (1769074449) [ 1461.094808] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 1474.305920] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1480.867954] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 1492.199197] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1497.753041] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 1501.448818] Lustre: 47565:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1509.872789] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1517.824303] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 1529.624788] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1537.314900] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 1546.695360] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1557.252176] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1582.198707] Lustre: server umount lustre-MDT0000 complete [ 1586.814472] LustreError: 46423:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769074597 with bad export cookie 10223810400771019900 [ 1586.819798] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1586.829809] LustreError: 46423:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1587.452655] Lustre: server umount lustre-MDT0001 complete [ 1602.311127] Lustre: server umount lustre-OST0000 complete [ 1617.108089] Lustre: server umount lustre-OST0001 complete [ 1629.763695] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 04:37:18 (1769074638) [ 1632.621860] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1635.153797] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 04:37:23 (1769074643) [ 1650.694878] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_hostid [ 1661.559253] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 1674.091327] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1683.542094] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1693.138688] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1693.266688] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1693.731594] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1693.743532] Lustre: Skipped 1 previous similar message [ 1693.800042] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1693.921786] Lustre: lustre-MDT0000: new disk, initializing [ 1694.013598] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1699.779738] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 1711.717383] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1722.095719] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1722.190188] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1722.437422] Lustre: lustre-OST0000: new disk, initializing [ 1722.441739] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1722.448165] Lustre: Skipped 1 previous similar message [ 1724.201814] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1724.207811] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1724.245872] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1729.164604] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 1739.001037] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1742.332687] Lustre: DEBUG MARKER: create test files [ 1785.825235] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1785.836713] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1785.851079] Lustre: Skipped 14 previous similar messages [ 1789.774581] Lustre: server umount lustre-OST0000 complete [ 1794.857840] Lustre: server umount lustre-MDT0000 complete [ 1808.318843] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 1812.476908] Key type lgssc unregistered [ 1812.839187] LNet: 55014:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1812.844446] LNetError: 55014:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1812.878373] LNet: Removed LNI 192.168.202.152@tcp [ 1813.995705] Key type .llcrypt unregistered [ 1813.999952] Key type ._llcrypt unregistered [ 1849.921633] Key type ._llcrypt registered [ 1849.925295] Key type .llcrypt registered [ 1850.134684] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 1852.193201] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1852.216168] alg: No test for adler32 (adler32-zlib) [ 1853.390916] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 1853.741040] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 1855.400383] Key type lgssc registered [ 1856.207870] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1866.085567] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1866.110422] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1867.904070] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1873.663628] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 1881.335174] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1890.876393] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1891.471694] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1892.465383] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1892.672051] Lustre: 57617:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1892.676599] Lustre: 57617:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1892.679455] Lustre: 57617:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1892.682484] Lustre: 57617:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1892.686403] Lustre: 57617:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1892.690310] Lustre: 57617:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1898.646115] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 1907.168590] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1913.242389] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1921.095992] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1933.799217] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1933.802257] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1933.812726] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1936.200802] LustreError: 58362:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1936.350991] Lustre: server umount lustre-OST0000 complete [ 1941.017986] LustreError: 58565:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1941.027834] LustreError: 58565:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1941.313423] Lustre: server umount lustre-MDT0000 complete [ 1953.383436] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 1956.593618] Key type lgssc unregistered [ 1956.924398] LNet: 59097:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1956.934067] LNetError: 59097:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1957.993728] LNet: Removed LNI 192.168.202.152@tcp [ 1958.925455] Key type .llcrypt unregistered [ 1958.927058] Key type ._llcrypt unregistered [ 1993.678781] Key type ._llcrypt registered [ 1993.684296] Key type .llcrypt registered [ 1993.910357] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 1995.694242] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1995.737452] alg: No test for adler32 (adler32-zlib) [ 1996.838193] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 1997.088907] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 1998.865882] Key type lgssc registered [ 1999.941949] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2009.473622] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2009.499990] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2011.240307] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2016.825290] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 2024.510900] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2034.003545] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2034.565628] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2036.591161] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 2036.651238] Lustre: 61804:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 2036.656722] Lustre: 61804:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 2036.667157] Lustre: 61804:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2036.682602] Lustre: 61804:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 2036.700327] Lustre: 61804:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2036.702503] Lustre: 61804:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2041.535701] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 2049.517520] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2055.503860] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 2098.154474] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2098.162269] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2098.190925] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2101.211208] LustreError: 62528:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 2101.439467] Lustre: server umount lustre-OST0000 complete [ 2106.240961] LustreError: 62730:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 2106.251906] LustreError: 62730:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 2106.474411] Lustre: server umount lustre-MDT0000 complete [ 2119.638159] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 2122.439325] Key type lgssc unregistered [ 2122.762797] LNet: 63262:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2122.771882] LNetError: 63262:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2122.809995] LNet: Removed LNI 192.168.202.152@tcp [ 2123.822878] Key type .llcrypt unregistered [ 2123.825415] Key type ._llcrypt unregistered [ 2146.679809] Key type ._llcrypt registered [ 2146.681417] Key type .llcrypt registered [ 2146.816809] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_hostid [ 2164.348706] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 2166.215669] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2166.298202] alg: No test for adler32 (adler32-zlib) [ 2167.477320] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 2167.762962] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 2169.544500] Key type lgssc registered [ 2170.900233] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2182.207152] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2190.541173] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2200.003125] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2209.812368] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2219.580382] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2219.642464] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2219.669600] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2221.111656] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2221.204580] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2221.344319] Lustre: lustre-MDT0000: new disk, initializing [ 2221.587667] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2221.647180] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2227.717433] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 2240.305280] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2240.416682] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2240.503753] Lustre: 66399:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2240.591694] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2240.598044] Lustre: Skipped 1 previous similar message [ 2240.709601] Lustre: lustre-MDT0001: new disk, initializing [ 2240.826679] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2240.900134] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2240.927809] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2245.996756] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 2255.035353] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2259.953935] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2269.019207] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2269.091543] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2269.439412] Lustre: lustre-OST0000: new disk, initializing [ 2269.444406] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2269.555039] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2276.021963] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2276.048858] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2276.218771] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2277.491205] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 2287.149184] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2293.457341] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2293.758872] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2299.372297] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2299.683248] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2307.048239] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2307.082547] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2307.084188] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2307.102498] Lustre: Skipped 1 previous similar message [ 2308.301481] LustreError: 69339:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2308.510979] Lustre: server umount lustre-OST0000 complete [ 2317.284308] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2317.292448] Lustre: Skipped 2 previous similar messages [ 2317.292690] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2319.212263] LustreError: 69542:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2319.220752] LustreError: 69542:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2319.569506] Lustre: server umount lustre-MDT0000 complete [ 2324.847871] LustreError: 66391:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769075335 with bad export cookie 11668219179438420205 [ 2324.864052] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2324.871759] LustreError: 66391:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2325.105076] LustreError: 69743:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2325.116131] LustreError: 69743:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2325.299106] Lustre: server umount lustre-MDT0001 complete [ 2336.259327] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 04:49:04 (1769075344) [ 2353.049550] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 2371.899062] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2402.790239] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_hostid [ 2414.189593] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 2426.732363] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2435.398673] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2444.133092] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2452.767203] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2463.452101] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2463.547737] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2463.853963] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2463.892865] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2463.998697] Lustre: lustre-MDT0000: new disk, initializing [ 2464.193070] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2464.245342] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2470.564212] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 2484.997175] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2485.088276] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2485.305082] Lustre: 74019:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2485.432069] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2485.439252] Lustre: Skipped 1 previous similar message [ 2485.534530] Lustre: lustre-MDT0001: new disk, initializing [ 2485.675853] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2485.732425] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2485.744721] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2491.554871] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 2500.777216] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2505.952287] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2514.696246] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2514.849812] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2515.265219] Lustre: lustre-OST0000: new disk, initializing [ 2515.277437] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2515.383567] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2522.203433] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2522.214633] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2522.321348] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2522.889538] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 2532.736176] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2538.826272] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2539.105925] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2544.680705] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2545.058835] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2548.196471] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2548.199090] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2548.206410] LustreError: Skipped 1 previous similar message [ 2548.212513] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2548.228122] Lustre: Skipped 1 previous similar message [ 2548.264255] Lustre: Skipped 2 previous similar messages [ 2553.324318] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2553.335914] Lustre: Skipped 1 previous similar message [ 2553.514127] LustreError: 76960:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2553.518279] LustreError: 76960:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 2553.796484] Lustre: server umount lustre-OST0000 complete [ 2558.444118] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2558.460354] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2558.462217] Lustre: Skipped 1 previous similar message [ 2558.479455] Lustre: Skipped 1 previous similar message [ 2564.732391] LustreError: 77163:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2564.745619] LustreError: 77163:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2565.154524] Lustre: server umount lustre-MDT0000 complete [ 2568.683512] LustreError: 74030:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2568.697597] LustreError: 74030:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 2570.343209] LustreError: 74011:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769075581 with bad export cookie 11668219179438421087 [ 2570.358599] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2570.367716] LustreError: 74011:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2570.836610] Lustre: server umount lustre-MDT0001 complete [ 2581.416235] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 04:53:09 (1769075589) [ 2589.372685] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2589.923289] LustreError: 78037:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2590.027972] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2595.302189] LustreError: 78037:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2595.316243] LustreError: 78037:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2596.376699] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 2599.403317] LustreError: 78037:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2604.518826] LustreError: 78037:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2606.278762] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2612.420932] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 2621.458966] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2626.029963] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2635.075036] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2635.535198] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2635.547719] Lustre: Skipped 1 previous similar message [ 2643.563738] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 2651.197825] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2655.683775] Lustre: 80378:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting 1b3a8a59-e8ac-402e-ab11-e774864719df at adminstrative request [ 2657.261622] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2657.281937] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2667.554770] Lustre: MGS: Client 1b3a8a59-e8ac-402e-ab11-e774864719df (at 0@lo) reconnecting [ 2667.569302] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xa1edd785d7d3b7d1 to 0xa1edd785d7d3ba6a [ 2667.585156] Lustre: MGC192.168.202.152@tcp: Connection restored to 0@lo (at 0@lo) [ 2695.076894] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2695.197320] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2695.414408] Lustre: lustre-OST0001: new disk, initializing [ 2695.419751] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2701.874204] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 2703.498639] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2703.513455] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2703.675938] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2710.948077] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2719.204716] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2719.227643] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2719.229624] LustreError: Skipped 1 previous similar message [ 2719.231907] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2719.231915] Lustre: Skipped 2 previous similar messages [ 2719.264821] Lustre: Skipped 1 previous similar message [ 2720.251439] LustreError: 81518:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 2720.259604] LustreError: 81518:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 2720.335923] Lustre: server umount lustre-OST0001 complete [ 2724.336051] LustreError: 79637:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2724.368457] LustreError: 79637:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2725.858041] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2728.933538] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2728.946523] Lustre: Skipped 1 previous similar message [ 2731.332879] Lustre: server umount lustre-OST0000 complete [ 2739.684408] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2739.702736] Lustre: Skipped 6 previous similar messages [ 2741.737242] LustreError: 81922:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 2741.756577] LustreError: 81922:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 2742.057676] Lustre: server umount lustre-MDT0000 complete [ 2744.803048] LustreError: 78036:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2744.824775] LustreError: 78036:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2747.108718] LustreError: 78017:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769075757 with bad export cookie 11668219179438422634 [ 2747.127467] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2747.138352] LustreError: 78017:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2747.740812] Lustre: server umount lustre-MDT0001 complete [ 2759.689563] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 04:56:07 (1769075767) [ 2762.176466] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2764.679236] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 04:56:12 (1769075772) [ 2767.321757] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2770.039570] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 04:56:18 (1769075778) [ 2772.620043] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2775.376282] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 04:56:23 (1769075783) [ 2777.709474] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2781.024617] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 04:56:28 (1769075788) [ 2783.057149] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2790.393266] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2792.263052] loop: module loaded [ 2792.292570] loop0: detected capacity change from 0 to 4294967296 [ 2792.411161] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2794.633312] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2800.597762] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2807.578755] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2818.495679] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2824.061305] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2829.506779] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2834.608712] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2842.800693] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2842.864512] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2842.897283] LustreError: 84804:0:(obd_config.c:783:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2842.904877] LustreError: 84804:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2842.910924] LustreError: 84804:0:(tgt_mount.c:2447:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2842.919969] LustreError: 84804:0:(super25.c:199:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2864.912220] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_hostid [ 2876.255853] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 2888.054335] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2896.124354] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2905.130146] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2914.315585] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2923.492758] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2923.623345] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2924.068257] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2924.176452] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2924.343807] Lustre: lustre-MDT0000: new disk, initializing [ 2924.455183] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2924.458165] Lustre: Skipped 1 previous similar message [ 2924.478926] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2930.682737] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 2944.159037] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2944.255319] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2944.474070] Lustre: 87651:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2944.951791] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2950.811471] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 2960.499707] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2965.477418] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2974.580590] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2974.657373] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2983.720394] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 2985.083526] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2985.374411] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2992.989785] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2999.507995] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2999.857398] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3007.001658] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3007.375324] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3011.044042] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3011.045989] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3011.059955] LustreError: Skipped 2 previous similar messages [ 3011.063077] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3011.103960] Lustre: Skipped 3 previous similar messages [ 3024.352156] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3024.423509] LustreError: 90594:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3024.426331] LustreError: 90594:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 3024.790651] Lustre: server umount lustre-OST0000 complete [ 3036.345614] Lustre: server umount lustre-MDT0000 complete [ 3036.642987] LustreError: 87663:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3036.654962] LustreError: 87663:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3041.271456] LustreError: 87644:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769076052 with bad export cookie 11668219179438423236 [ 3041.288891] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3041.962648] Lustre: server umount lustre-MDT0001 complete [ 3055.968676] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 05:01:03 (1769076063) [ 3064.680610] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3065.567677] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 3065.576791] Lustre: Skipped 2 previous similar messages [ 3070.515355] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 3081.544696] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3088.748434] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 3096.023778] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3102.057686] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3113.007243] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3123.617693] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 3132.640675] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3138.864616] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3139.185846] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3144.932347] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3145.394788] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3156.541965] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3156.707319] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3157.162480] Lustre: lustre-OST0001: new disk, initializing [ 3157.172564] Lustre: Skipped 2 previous similar messages [ 3157.183192] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3157.196543] Lustre: Skipped 3 previous similar messages [ 3164.286101] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3164.299042] Lustre: Skipped 2 previous similar messages [ 3164.314275] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3164.437263] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3165.906480] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 3175.983319] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 3181.862878] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 3182.088085] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3187.659323] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 3187.852288] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3207.603185] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3238.283675] Lustre: Failing over lustre-MDT0000 [ 3238.702632] LustreError: 96608:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 3238.710839] LustreError: 96608:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 3238.905620] Lustre: server umount lustre-MDT0000 complete [ 3240.929700] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3240.937683] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3240.945569] Lustre: Skipped 2 previous similar messages [ 3240.949290] LustreError: 92578:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3240.958760] LustreError: 92578:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 3257.824158] Lustre: 64605:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1769076252/real 1769076252] req@000000002c485349 x1855009774139008/t0(0) o400->MGC192.168.202.152@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1769076268 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3257.843504] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3266.675927] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3266.679091] LDISKFS-fs (dm-0): recovery complete [ 3266.699626] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3267.872165] LustreError: 97328:0:(import.c:337:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 3267.876453] LustreError: 97328:0:(import.c:361:ptlrpc_invalidate_import()) @@@ still on sending list req@0000000045525e2e x1855009774144256/t0(0) o250->MGC192.168.202.152@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1769076278 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3267.885332] LustreError: 97328:0:(import.c:371:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 3268.080895] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0xa1edd785d7d484d0 [ 3268.092412] Lustre: MGC192.168.202.152@tcp: Connection restored to 0@lo (at 0@lo) [ 3268.553738] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3273.585410] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3273.587475] Lustre: 91680:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3273.761469] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3273.928630] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3274.280406] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3274.296144] Lustre: 97360:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3274.316025] Lustre: 97360:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 7 previous similar messages [ 3274.666250] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3274.673435] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 3274.750671] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 3274.997815] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3275.017906] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 3275.384900] Lustre: 97360:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3275.398143] Lustre: 97360:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 2 previous similar messages [ 3275.731296] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3275.746166] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 2 previous similar messages [ 3276.092149] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3276.104884] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 2 previous similar messages [ 3277.426830] Lustre: 97360:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3277.444830] Lustre: 97360:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 5 previous similar messages [ 3277.777753] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3277.801753] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 5 previous similar messages [ 3278.146635] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3278.170939] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 5 previous similar messages [ 3281.451607] Lustre: 97360:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3281.465710] Lustre: 97360:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 11 previous similar messages [ 3281.802286] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3281.815152] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 11 previous similar messages [ 3282.153737] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3282.172129] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 11 previous similar messages [ 3289.537291] Lustre: 97360:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3289.552669] Lustre: 97360:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 24 previous similar messages [ 3289.879333] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3289.916980] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 24 previous similar messages [ 3290.256282] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3290.268058] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 24 previous similar messages [ 3305.686250] Lustre: 97360:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3305.695933] Lustre: 97360:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 49 previous similar messages [ 3306.037212] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3306.039992] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 49 previous similar messages [ 3306.368200] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3306.375453] LustreError: 97360:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 49 previous similar messages [ 3333.208547] Lustre: 97360:0:(ldlm_lib.c:2377:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3333.230062] Lustre: 97360:0:(ldlm_lib.c:2387:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3333.242332] Lustre: 97360:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 1cbec7b9-e922-4e04-8c0d-cde2def93631@192.168.202.52@tcp [ 3333.272561] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3333.294642] Lustre: 97360:0:(ldlm_lib.c:1897:abort_req_replay_queue()) @@@ aborted: req@00000000d666a4cb x1855009755044480/t0(8589934780) o35->1cbec7b9-e922-4e04-8c0d-cde2def93631@192.168.202.52@tcp:369/0 lens 392/0 e 0 to 0 dl 1769076354 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3333.351928] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3333.384099] Lustre: lustre-MDT0000: Denying connection for new client 1cbec7b9-e922-4e04-8c0d-cde2def93631 (at 192.168.202.52@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3333.410719] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3333.473242] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3333.479268] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3333.489718] Lustre: Skipped 2 previous similar messages [ 3333.572146] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 3333.576465] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 3355.115063] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3355.142390] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3355.156193] Lustre: Skipped 3 previous similar messages [ 3355.170793] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3355.175199] Lustre: Skipped 7 previous similar messages [ 3360.732082] Lustre: server umount lustre-OST0000 complete [ 3370.978957] LustreError: 94970:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3370.988937] LustreError: 94970:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 45 previous similar messages [ 3371.204220] LustreError: 98224:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 3371.211471] LustreError: 98224:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 3371.323373] Lustre: server umount lustre-OST0001 complete [ 3382.713152] Lustre: server umount lustre-MDT0000 complete [ 3387.083602] LustreError: 94692:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769076397 with bad export cookie 11668219179438474448 [ 3387.091057] LustreError: 94692:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3387.103212] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3387.777801] Lustre: server umount lustre-MDT0001 complete [ 3398.485567] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 05:06:46 (1769076406) [ 3407.502579] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3407.510155] Lustre: *** cfs_fail_loc=197, val=0*** [ 3423.136967] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3428.256671] LustreError: 99320:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.152@tcp: failed processing log, type 4: rc = -110 [ 3433.407673] Lustre: *** cfs_fail_loc=197, val=0*** [ 3433.417557] Lustre: Skipped 65 previous similar messages [ 3453.984356] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3453.990891] Lustre: Skipped 4 previous similar messages [ 3461.085286] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 3468.003526] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3470.429550] Lustre: server umount lustre-OST0000 complete [ 3480.808528] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 05:08:09 (1769076489) [ 3504.206760] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3509.909238] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 3519.374424] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3523.768592] LustreError: 100861:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3523.783931] LustreError: 100861:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3523.798041] LustreError: 100861:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 3524.391564] Lustre: server umount lustre-MDT0000 complete [ 3542.016804] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3563.230164] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_hostid [ 3575.588623] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 3590.953509] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3601.087774] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3610.655822] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3618.582550] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3626.810371] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3626.889765] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3627.075929] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3627.148793] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3627.232175] Lustre: lustre-MDT0000: new disk, initializing [ 3627.405336] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3632.958372] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 3645.933332] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3646.041643] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3646.135875] Lustre: 104908:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3646.387906] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3651.132553] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 3660.945581] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3667.219432] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3676.939275] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3677.055361] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3685.011661] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 3686.218919] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3694.976432] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3701.321542] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3701.520182] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3706.678743] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3706.955442] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3711.969403] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3711.984892] LustreError: Skipped 2 previous similar messages [ 3711.993977] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3712.025193] Lustre: Skipped 5 previous similar messages [ 3712.046471] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3712.049454] Lustre: Skipped 8 previous similar messages [ 3715.423898] LustreError: 107848:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3715.427069] LustreError: 107848:0:(obd_class.h:479:obd_check_dev()) Skipped 37 previous similar messages [ 3715.693566] Lustre: server umount lustre-OST0000 complete [ 3727.502409] Lustre: server umount lustre-MDT0000 complete [ 3731.426643] LustreError: 104919:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3731.446079] LustreError: 104919:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 3732.347617] LustreError: 104900:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769076743 with bad export cookie 11668219179438485130 [ 3732.366480] LustreError: 104900:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3732.384527] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3732.999338] Lustre: server umount lustre-MDT0001 complete [ 3744.082985] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 05:12:32 (1769076752) [ 3756.306775] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 3760.319524] Key type lgssc unregistered [ 3760.744890] LNet: 108983:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3760.756390] LNetError: 108983:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3761.771674] LNet: Removed LNI 192.168.202.152@tcp [ 3762.886165] Key type .llcrypt unregistered [ 3762.890953] Key type ._llcrypt unregistered [ 3786.294873] Key type ._llcrypt registered [ 3786.297621] Key type .llcrypt registered [ 3786.479396] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_hostid [ 3804.533937] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 3805.634596] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3805.669515] alg: No test for adler32 (adler32-zlib) [ 3807.202735] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 3807.448571] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 3809.136237] Key type lgssc registered [ 3810.622765] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3822.572841] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3829.894175] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3838.839456] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3848.938340] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3857.473518] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3866.876400] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3866.955087] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3866.969432] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3868.349191] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3868.387548] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3868.531587] Lustre: lustre-MDT0000: new disk, initializing [ 3868.694947] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3868.746582] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3875.398636] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 3889.288066] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3889.393056] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3889.806966] Lustre: lustre-OST0000: new disk, initializing [ 3889.813109] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3889.823449] Lustre: Skipped 1 previous similar message [ 3889.950965] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3897.535819] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3897.546567] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3897.600400] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3898.238194] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 3913.081599] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3913.209602] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3913.293865] Lustre: lustre-OST0001: new disk, initializing [ 3913.298062] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3913.444060] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3919.217880] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3919.235473] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3919.285733] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3922.230141] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 3934.696194] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3934.700484] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3934.702725] Lustre: Skipped 1 previous similar message [ 3934.723487] Lustre: Skipped 1 previous similar message [ 3935.432885] LustreError: 114568:0:(obd_class.h:479:obd_check_dev()) Device 17 not setup [ 3935.846133] Lustre: server umount lustre-MDT0000 complete [ 3943.439833] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3949.009083] LustreError: 112325:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769076959 with bad export cookie 3966532762510969838 [ 3949.024406] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3949.030221] LustreError: 112325:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3949.070169] LustreError: 115163:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 3949.075269] LustreError: 115163:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3949.115211] Lustre: server umount lustre-OST0000 complete [ 3953.685212] LustreError: 115373:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 3953.700524] LustreError: 115373:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 3953.838021] Lustre: server umount lustre-OST0001 complete [ 3965.620848] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 05:16:13 (1769076973) [ 3975.979524] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 3985.611164] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3986.700278] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3992.763558] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3994.162377] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4000.706644] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 05:16:49 (1769077009) [ 4014.983517] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4017.340778] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4019.511862] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4021.929865] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4022.970486] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4025.293391] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4046.695067] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_hostid [ 4058.732082] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 4071.621701] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4080.254942] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4088.813639] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4097.213448] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4105.448236] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4105.515949] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4105.744701] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4105.772807] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4105.843248] Lustre: lustre-MDT0000: new disk, initializing [ 4105.980974] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4105.997085] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4110.663102] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 4124.890574] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4124.926831] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4125.010754] Lustre: 120707:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4125.055214] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4125.060631] Lustre: Skipped 1 previous similar message [ 4125.107957] Lustre: lustre-MDT0001: new disk, initializing [ 4125.171673] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4125.221104] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4125.231392] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4129.943715] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 4140.187991] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4145.093978] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4154.674327] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4154.816079] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4155.098119] Lustre: lustre-OST0000: new disk, initializing [ 4155.104659] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4155.250848] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4163.181343] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4163.201177] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4163.363054] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4163.807769] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 4173.888222] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4180.348760] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4180.603242] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4186.771848] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4187.248756] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4194.278918] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4194.282242] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4194.286396] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4194.293759] LustreError: Skipped 1 previous similar message [ 4194.329790] Lustre: Skipped 1 previous similar message [ 4199.396713] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4199.416201] Lustre: Skipped 2 previous similar messages [ 4204.000312] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4204.041096] LustreError: 123654:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4204.043887] LustreError: 123654:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4204.245684] Lustre: server umount lustre-OST0000 complete [ 4209.637558] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4209.660130] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4209.666545] Lustre: Skipped 1 previous similar message [ 4214.757443] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4214.767035] Lustre: Skipped 1 previous similar message [ 4215.202811] LustreError: 123857:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 4215.211237] LustreError: 123857:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4215.349392] Lustre: server umount lustre-MDT0000 complete [ 4219.684919] LustreError: 120700:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769077230 with bad export cookie 3966532762510971070 [ 4219.685423] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4219.699355] LustreError: 120700:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4220.219973] Lustre: server umount lustre-MDT0001 complete [ 4230.867532] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 05:20:39 (1769077239) [ 4239.557506] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4240.316215] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4245.481385] LustreError: 124729:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4245.500753] LustreError: 124729:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4246.377534] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 4250.598141] LustreError: 124729:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4255.729215] LustreError: 124729:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4257.362409] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4263.770274] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 4269.679720] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4274.168455] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4282.464444] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4282.672159] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4282.677191] Lustre: Skipped 1 previous similar message [ 4289.842458] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 4296.982432] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4302.386679] Lustre: *** cfs_fail_loc=159, val=0*** [ 4318.640751] Lustre: lustre-MDT0000: Client 34f51f2e-515a-4f70-b06b-0a32cb346bb1 (at 192.168.202.52@tcp) reconnecting [ 4320.444615] Lustre: *** cfs_fail_loc=159, val=0*** [ 4320.453150] Lustre: Skipped 5 previous similar messages [ 4336.069215] Lustre: lustre-MDT0000: Client 34f51f2e-515a-4f70-b06b-0a32cb346bb1 (at 192.168.202.52@tcp) reconnecting [ 4345.318691] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4345.328715] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4345.353535] Lustre: Skipped 1 previous similar message [ 4345.361791] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4347.395472] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4348.444419] LustreError: 127222:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4348.463386] LustreError: 127222:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 4348.798544] Lustre: server umount lustre-OST0000 complete [ 4355.043109] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4355.055068] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4355.084896] Lustre: Skipped 1 previous similar message [ 4360.933872] Lustre: server umount lustre-MDT0000 complete [ 4362.733716] LustreError: 124728:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4362.750857] LustreError: 124728:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4365.911968] LustreError: 124710:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769077376 with bad export cookie 3966532762510971952 [ 4365.917797] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4365.923045] LustreError: 124710:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 4366.114798] LustreError: 127625:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4366.129767] LustreError: 127625:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 4366.330123] Lustre: server umount lustre-MDT0001 complete [ 4381.133771] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 4384.680178] Key type lgssc unregistered [ 4385.022272] LNet: 128158:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4385.033308] LNetError: 128158:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4385.061499] LNet: Removed LNI 192.168.202.152@tcp [ 4386.311215] Key type .llcrypt unregistered [ 4386.312960] Key type ._llcrypt unregistered [ 4415.246867] Key type ._llcrypt registered [ 4415.249223] Key type .llcrypt registered [ 4415.381987] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 4416.249956] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4416.310763] alg: No test for adler32 (adler32-zlib) [ 4417.308996] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 4417.530780] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 4419.232208] Key type lgssc registered [ 4420.814866] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4431.015781] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4431.039885] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4432.814424] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4439.511968] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 4443.107957] LustreError: 129553:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4443.124908] LustreError: 129553:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4448.229934] LustreError: 129553:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4448.939606] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4449.552719] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4454.919269] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 4461.866894] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4466.823632] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4475.661657] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4476.168220] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4478.203697] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4483.762798] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 4490.729668] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4504.641552] Lustre: *** cfs_fail_loc=159, val=0*** [ 4519.836371] Lustre: lustre-MDT0000: Client ff72eb98-35cc-4797-b723-d69d57257681 (at 192.168.202.52@tcp) reconnecting [ 4522.701729] Lustre: *** cfs_fail_loc=159, val=0*** [ 4538.260605] Lustre: lustre-MDT0001: Client ff72eb98-35cc-4797-b723-d69d57257681 (at 192.168.202.52@tcp) reconnecting [ 4540.170404] Lustre: *** cfs_fail_loc=159, val=0*** [ 4540.176823] Lustre: Skipped 3 previous similar messages [ 4556.727653] Lustre: lustre-MDT0001: Client ff72eb98-35cc-4797-b723-d69d57257681 (at 192.168.202.52@tcp) reconnecting [ 4564.562862] Lustre: 132258:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4568.937174] Lustre: *** cfs_fail_loc=159, val=0*** [ 4568.942539] Lustre: Skipped 3 previous similar messages [ 4585.336752] Lustre: lustre-MDT0000: Client 7a9cb01b-ca32-48e4-9079-34e040966bf5 (at 192.168.202.52@tcp) reconnecting [ 4587.051258] Lustre: *** cfs_fail_loc=159, val=0*** [ 4587.057663] Lustre: Skipped 15 previous similar messages [ 4602.773275] Lustre: lustre-MDT0000: Client 7a9cb01b-ca32-48e4-9079-34e040966bf5 (at 192.168.202.52@tcp) reconnecting [ 4604.394297] Lustre: 132561:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4626.916465] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4626.927510] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4626.940248] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4627.940146] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4627.943635] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4629.364420] LustreError: 132688:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4629.509897] Lustre: server umount lustre-OST0000 complete [ 4634.081445] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4634.086845] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4634.118335] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4634.120266] Lustre: Skipped 1 previous similar message [ 4638.180967] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4638.181314] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4639.616835] LustreError: 132889:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4639.633357] LustreError: 132889:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4639.961230] Lustre: server umount lustre-MDT0000 complete [ 4643.300862] LustreError: 129553:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4643.320812] LustreError: 129553:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4643.822535] LustreError: 132576:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769077654 with bad export cookie 4249611352598703019 [ 4643.833139] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4643.841265] LustreError: 132576:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4643.984579] LustreError: 133091:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4643.992802] LustreError: 133091:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4644.167276] Lustre: server umount lustre-MDT0001 complete [ 4656.832832] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 4659.946932] Key type lgssc unregistered [ 4660.228778] LNet: 133623:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4660.237509] LNetError: 133623:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4661.288504] LNet: Removed LNI 192.168.202.152@tcp [ 4662.243447] Key type .llcrypt unregistered [ 4662.245552] Key type ._llcrypt unregistered [ 4685.255374] Key type ._llcrypt registered [ 4685.257062] Key type .llcrypt registered [ 4685.350969] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 4686.087575] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4686.104097] alg: No test for adler32 (adler32-zlib) [ 4687.124389] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 4687.277054] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 4689.008189] Key type lgssc registered [ 4690.011214] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4700.332085] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4700.389776] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4702.191086] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4707.951220] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 4711.399310] LustreError: 135023:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4711.413074] LustreError: 135023:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4716.520194] LustreError: 135023:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4718.205613] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4718.733444] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4724.472367] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 4730.998152] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4735.786302] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4743.486805] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4743.654179] LustreError: 136625:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4743.885349] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4747.002839] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4747.010441] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4751.107937] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 4757.173435] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4764.480194] Lustre: 137275:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4767.975123] Lustre: 137323:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4769.764064] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4770.650809] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4774.545134] Lustre: 137470:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4775.369855] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4776.416208] Lustre: 137568:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4783.077842] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4783.094922] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4783.113597] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4787.781180] LustreError: 137676:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4787.965218] Lustre: server umount lustre-OST0000 complete [ 4793.316144] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4793.327055] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4793.334763] Lustre: Skipped 2 previous similar messages [ 4793.349850] Lustre: Skipped 2 previous similar messages [ 4798.433923] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4798.446340] Lustre: Skipped 1 previous similar message [ 4803.563061] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4803.578193] Lustre: Skipped 1 previous similar message [ 4806.112193] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4806.189822] LustreError: 137877:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4806.202211] LustreError: 137877:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4806.277273] Lustre: server umount lustre-MDT0000 complete [ 4808.673275] LustreError: 135024:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4811.293499] LustreError: 135005:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769077822 with bad export cookie 18083727670629728902 [ 4811.298362] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4811.315426] LustreError: 135005:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4811.449615] LustreError: 138081:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4811.454938] LustreError: 138081:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4811.626844] Lustre: server umount lustre-MDT0001 complete [ 4821.928169] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 4825.313693] Key type lgssc unregistered [ 4825.640329] LNet: 138612:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4825.656644] LNetError: 138612:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4825.674686] LNet: Removed LNI 192.168.202.152@tcp [ 4826.850165] Key type .llcrypt unregistered [ 4826.851560] Key type ._llcrypt unregistered [ 4854.305575] Key type ._llcrypt registered [ 4854.308586] Key type .llcrypt registered [ 4854.513515] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 4855.721920] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4855.898956] alg: No test for adler32 (adler32-zlib) [ 4856.893258] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 4857.037614] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 4858.672434] Key type lgssc registered [ 4859.863604] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4872.656669] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4872.680569] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4874.775946] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4882.398729] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 4884.980899] LustreError: 140009:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4885.009545] LustreError: 140009:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4890.085363] LustreError: 140009:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4894.379817] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4894.987446] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4900.290329] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 4907.445461] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4911.200115] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4918.767125] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4919.108800] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4920.119499] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4921.156396] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4925.458836] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 4933.866414] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4941.480253] Lustre: *** cfs_fail_loc=159, val=0*** [ 4957.113332] Lustre: lustre-MDT0000: Client 883080e9-d3bf-49a9-8cd8-5864bad304e2 (at 192.168.202.52@tcp) reconnecting [ 4976.619406] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4976.627615] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4976.642806] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4977.636742] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4977.644620] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4977.646539] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4981.645175] LustreError: 142394:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4982.179622] Lustre: server umount lustre-OST0000 complete [ 4987.899389] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4987.918677] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4987.930039] Lustre: Skipped 2 previous similar messages [ 4992.995735] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4992.998142] Lustre: Skipped 1 previous similar message [ 4993.122739] LustreError: 142596:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4993.133830] LustreError: 142596:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4993.359516] Lustre: server umount lustre-MDT0000 complete [ 4997.128041] LustreError: 142291:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769078007 with bad export cookie 13738981827616555926 [ 4997.156601] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4997.292503] LustreError: 142797:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4997.305336] LustreError: 142797:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4997.445569] Lustre: server umount lustre-MDT0001 complete [ 5008.116566] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 5011.240303] Key type lgssc unregistered [ 5011.634886] LNet: 143329:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5011.641364] LNetError: 143329:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5011.663332] LNet: Removed LNI 192.168.202.152@tcp [ 5012.799990] Key type .llcrypt unregistered [ 5012.806553] Key type ._llcrypt unregistered [ 5038.485466] Key type ._llcrypt registered [ 5038.495553] Key type .llcrypt registered [ 5038.651833] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 5039.787819] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5040.029082] alg: No test for adler32 (adler32-zlib) [ 5041.222401] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 5041.476665] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 5043.144186] Key type lgssc registered [ 5044.631987] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5054.395672] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5054.418088] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5056.091213] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5061.395785] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 5065.198065] LustreError: 144750:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5065.212903] LustreError: 144750:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5070.311685] LustreError: 144750:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5070.699849] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5071.186738] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5076.393872] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 5082.663253] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5087.467438] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5095.261372] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5095.708846] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5096.781198] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 5100.937388] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 5102.641099] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 5109.694662] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5130.866570] Lustre: lustre-OST0000: evicting 6315bf0b-36a9-48ba-9ffe-c92d6b221e1a (at 192.168.202.52@tcp) by administrative request [ 5130.878223] Lustre: Skipped 1 previous similar message [ 5162.465848] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5162.477428] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5162.478249] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5162.495057] Lustre: Skipped 1 previous similar message [ 5167.590064] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5167.608444] Lustre: Skipped 2 previous similar messages [ 5172.710558] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5172.718937] Lustre: Skipped 1 previous similar message [ 5174.240218] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5174.291163] LustreError: 147534:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5174.441190] Lustre: server umount lustre-OST0000 complete [ 5178.850533] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5178.859794] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5178.872778] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5182.945691] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5182.950410] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5193.184507] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5193.186456] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5193.200726] Lustre: Skipped 4 previous similar messages [ 5193.310127] LustreError: 147736:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 5193.328426] LustreError: 147736:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5193.574941] Lustre: server umount lustre-MDT0000 complete [ 5198.067929] LustreError: 147737:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769078208 with bad export cookie 8338072458967868104 [ 5198.072131] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5198.090144] LustreError: 147737:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5198.312448] LustreError: 147941:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5198.323391] LustreError: 147941:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5198.495863] Lustre: server umount lustre-MDT0001 complete [ 5210.526938] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 5213.897502] Key type lgssc unregistered [ 5214.316232] LNet: 148473:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5214.334990] LNetError: 148473:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5214.391611] LNet: Removed LNI 192.168.202.152@tcp [ 5215.681921] Key type .llcrypt unregistered [ 5215.683114] Key type ._llcrypt unregistered [ 5302.295644] Key type ._llcrypt registered [ 5302.303703] Key type .llcrypt registered [ 5302.514793] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 5305.111854] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5305.238157] alg: No test for adler32 (adler32-zlib) [ 5306.707845] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 5307.280231] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 5309.104614] Key type lgssc registered [ 5310.788215] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5320.450219] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5320.488547] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5322.391097] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5328.158238] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 5332.973597] LustreError: 150950:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5333.007950] LustreError: 150950:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5338.086888] LustreError: 150950:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5341.759865] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5342.133323] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5350.074820] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 5357.050658] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5361.583837] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5370.829212] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5371.137159] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5378.594842] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 5386.021742] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5386.560601] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5392.719758] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5405.574796] LustreError: 153230:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5405.869614] Lustre: server umount lustre-OST0000 complete [ 5413.348893] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5413.357714] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5413.370134] Lustre: Skipped 1 previous similar message [ 5413.379166] Lustre: Skipped 1 previous similar message [ 5418.465461] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5418.468580] Lustre: Skipped 1 previous similar message [ 5423.587687] Lustre: 150263:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1769078418/real 1769078418] req@00000000b1346a65 x1855013066407296/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1769078434 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 5423.591763] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5423.632967] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5423.633009] Lustre: Skipped 1 previous similar message [ 5424.608245] Lustre: 150261:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1769078419/real 1769078419] req@0000000099d29e41 x1855013066408064/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1769078435 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 5426.144253] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5426.272695] LustreError: 153432:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 5426.279457] LustreError: 153432:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5426.438306] Lustre: server umount lustre-MDT0000 complete [ 5428.705175] LustreError: 150950:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5428.722284] LustreError: 150950:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5429.796677] Lustre: 150261:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1769078424/real 1769078424] req@00000000b6fcd04f x1855013066409984/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1769078440 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 5431.649376] LustreError: 150931:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769078442 with bad export cookie 1239145303540386936 [ 5431.650549] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5431.662319] LustreError: 150931:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5431.873137] LustreError: 153635:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5431.880275] LustreError: 153635:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5432.221363] Lustre: server umount lustre-MDT0001 complete [ 5445.931384] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 5449.018181] Key type lgssc unregistered [ 5449.393622] LNet: 154168:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5449.409972] LNetError: 154168:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5449.425769] LNet: Removed LNI 192.168.202.152@tcp [ 5450.308084] Key type .llcrypt unregistered [ 5450.310325] Key type ._llcrypt unregistered [ 5478.446636] Key type ._llcrypt registered [ 5478.448950] Key type .llcrypt registered [ 5478.534846] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 5479.867798] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5480.049606] alg: No test for adler32 (adler32-zlib) [ 5481.161634] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 5481.487114] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 5483.200210] Key type lgssc registered [ 5484.588240] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5494.125826] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5504.720649] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 05:41:52 (1769078512) [ 5525.112392] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_hostid [ 5532.825763] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 5547.989021] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5558.913486] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5565.777668] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5574.654477] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5586.070798] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5586.214717] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5586.234584] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5587.796538] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5587.909635] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5588.211250] Lustre: lustre-MDT0000: new disk, initializing [ 5588.483315] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5588.591837] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5596.385532] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 5612.375600] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5612.495152] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5612.634409] Lustre: 158396:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5612.722449] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5612.728224] Lustre: Skipped 1 previous similar message [ 5612.821584] Lustre: lustre-MDT0001: new disk, initializing [ 5612.961914] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5613.029251] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5613.044712] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5618.737889] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 5628.387832] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5633.020659] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5641.702120] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5641.798733] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5642.186773] Lustre: lustre-OST0000: new disk, initializing [ 5642.193479] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5642.269201] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5648.517789] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5648.527379] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5648.727554] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5650.073455] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 5658.732445] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5674.470085] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5674.470947] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5674.481224] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5677.294196] LustreError: 160988:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5677.596349] Lustre: server umount lustre-OST0000 complete [ 5684.708702] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5684.716571] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5684.736894] Lustre: Skipped 2 previous similar messages [ 5684.751612] Lustre: Skipped 2 previous similar messages [ 5689.845443] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5689.858745] Lustre: Skipped 1 previous similar message [ 5690.040377] LustreError: 161190:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 5690.051994] LustreError: 161190:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5690.500419] Lustre: server umount lustre-MDT0000 complete [ 5695.768702] LustreError: 158388:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769078706 with bad export cookie 14273515893852598984 [ 5695.779673] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5695.965588] LustreError: 161392:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5695.974530] LustreError: 161392:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5696.229366] Lustre: server umount lustre-MDT0001 complete [ 5710.599228] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 5714.579510] Key type lgssc unregistered [ 5715.041413] LNet: 161923:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5715.047335] LNetError: 161923:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5715.065993] LNet: Removed LNI 192.168.202.152@tcp [ 5716.242332] Key type .llcrypt unregistered [ 5716.244661] Key type ._llcrypt unregistered [ 5743.977586] Key type ._llcrypt registered [ 5743.979611] Key type .llcrypt registered [ 5744.102486] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 5745.991205] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5746.025250] alg: No test for adler32 (adler32-zlib) [ 5747.141713] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 5747.379762] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 5749.136995] Key type lgssc registered [ 5750.509527] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5759.550332] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5759.572579] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5761.199168] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5766.209161] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 5770.725963] LustreError: 163323:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5770.752283] LustreError: 163323:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5775.846406] LustreError: 163323:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5776.691405] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5777.125060] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5781.864376] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 5789.171747] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5793.206916] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5801.088937] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5801.514652] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5805.617959] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5808.858947] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 5814.673409] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5821.218916] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5821.244393] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5821.267895] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5821.275966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.294616] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5822.474446] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5822.490063] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5822.509679] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5822.520704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.547859] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5823.703957] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5823.727894] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5823.747941] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5823.754493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.763362] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5824.859360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.915922] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5825.923258] Lustre: Skipped 1 previous similar message [ 5825.954989] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5825.963925] Lustre: Skipped 1 previous similar message [ 5825.968946] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5825.972770] Lustre: Skipped 1 previous similar message [ 5825.976974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.985644] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5825.996036] Lustre: Skipped 1 previous similar message [ 5827.230260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.424135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.366100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.363311] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5830.372857] Lustre: Skipped 3 previous similar messages [ 5830.391298] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5830.402083] Lustre: Skipped 3 previous similar messages [ 5830.404856] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5830.409709] Lustre: Skipped 3 previous similar messages [ 5830.412825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.418671] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5830.428857] Lustre: Skipped 3 previous similar messages [ 5831.641589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.842505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.876386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.194410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.680863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.249940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.403504] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5839.414668] Lustre: Skipped 6 previous similar messages [ 5839.445792] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5839.456643] Lustre: Skipped 6 previous similar messages [ 5839.462029] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5839.467728] Lustre: Skipped 6 previous similar messages [ 5839.472918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.482990] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5839.492744] Lustre: Skipped 6 previous similar messages [ 5841.063875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.535045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.718639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.091792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.611610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.457325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.264588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.826776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.004490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.125670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.178162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.170508] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5856.186201] Lustre: Skipped 11 previous similar messages [ 5856.206966] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5856.227785] Lustre: Skipped 11 previous similar messages [ 5856.236051] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5856.238990] Lustre: Skipped 11 previous similar messages [ 5856.250667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.271785] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5856.274555] Lustre: Skipped 11 previous similar messages [ 5857.351032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.445197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.757573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.828737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.802562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.036541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.943166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.192949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.435319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.790460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.862682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.926763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.182531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.181253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.333508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.429938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.597544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.724234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.678023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.690615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.559834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.532225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.468629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.276162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.414223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.636937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.615282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.681389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.903266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.212986] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5889.216842] Lustre: Skipped 29 previous similar messages [ 5889.235386] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5889.240346] Lustre: Skipped 29 previous similar messages [ 5889.243600] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5889.246731] Lustre: Skipped 29 previous similar messages [ 5889.249668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.257338] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5889.259934] Lustre: Skipped 29 previous similar messages [ 5890.378699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.551275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.494070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.777981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.004635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.073950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.265635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.922449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.955803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.831947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.251783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.506258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.617337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.502688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.870511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.059987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.876990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.178345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.808950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.620621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.215631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.635430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.878481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.280968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.386948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.479264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.854724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.055599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.356913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.795773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.954784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.568243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.115809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.462163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.940609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.288624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.437181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.502178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.985269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.253321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.768409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.128089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.680527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.246836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.541785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.793680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.024823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.356630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.653174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.588042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.188293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.686647] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5954.704781] Lustre: Skipped 51 previous similar messages [ 5954.725945] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5954.734090] Lustre: Skipped 51 previous similar messages [ 5954.739389] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5954.742285] Lustre: Skipped 51 previous similar messages [ 5954.747037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.756812] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5954.763233] Lustre: Skipped 51 previous similar messages [ 5956.046194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.335968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.312926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.518307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.652106] LustreError: 163343:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5960.665594] LustreError: 163343:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5960.679476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.742864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.657769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.824025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.949552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.197298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.522189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.881621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.997842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.978199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.132308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.877418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.830903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.775319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.135366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.265586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.093338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.158867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.204386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.509422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.001594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.128315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.388192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.592838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.476938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.273578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.278812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.283943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.408798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.814170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.990519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.309620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.244247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.335756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.588255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.146266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.333216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.423202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.502871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.522061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.453348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.452416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.643500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.719970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.133436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.334029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.438766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.402495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.610299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.609689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.125316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.554285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.942616] LustreError: 163343:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6019.948629] LustreError: 163343:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6019.959221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.214694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.233649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.234334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.209276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.505103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.826805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.017571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.188963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.272511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.216266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.380877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.821606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.177834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.257851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.313812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.291387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.120193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.093473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.287838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.121373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.100974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.046435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.213831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.530553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.617971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.611119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.741427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.916314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.003661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.403134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.614347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.884575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.047669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.104613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.078773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.043612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.295168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.638406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.715735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.877589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.013158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.418210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.776555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.044737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.167234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.422899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.646731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.774498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.940166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.280964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.415085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.698093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.901357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.878202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.005655] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6083.010704] Lustre: Skipped 111 previous similar messages [ 6083.034953] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6083.042455] Lustre: Skipped 111 previous similar messages [ 6083.045911] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6083.050399] Lustre: Skipped 111 previous similar messages [ 6083.053516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.067636] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6083.070607] Lustre: Skipped 111 previous similar messages [ 6084.052419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.185459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.405870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.681731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.942713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.224810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.445641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.706238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.946959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.120853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.348245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.491020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.643291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.821665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.004335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.927890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.097327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.109147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.072823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.034540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.050953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.042090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.147587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.174168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.327800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.281691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.492420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.637867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.216478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.445589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.706710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.768168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.999660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.038077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.146514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.309207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.516155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.622490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.935380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.156950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.231248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.552835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.679865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.019628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.198423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.212315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.563841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.409848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.484942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.502538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.578657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.799868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.806674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.957983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.024290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.200910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.133656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.116754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.189140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.198677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.033232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.945469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.811170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.780068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.868594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.032850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.067676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.335723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.673129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.726200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.921657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.091504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.300251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.385083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.537917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.559081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.606464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.655052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.732675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.914697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.095794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.304552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.406715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.627256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.803285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.088582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.249702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.399060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.251241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.533222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.783540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.857976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.997127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.146964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.074426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.070166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.069544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.022448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.116278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.198033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.254695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.377839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.410720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.355071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.302905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.064568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.941288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.734925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.652059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.555240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.595970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.488426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.610765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.481809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.382957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.219444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.404264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.762776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.961562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.037944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.030249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.067915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.183240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.982576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.863367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.747079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.718325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.470448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.490470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.520051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.724358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.008584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.846866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.013455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.981208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.237549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.314826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.265631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.221943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.086951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.090791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.019477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.133228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.990212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.002337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.161154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.012707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.052517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.967532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.956577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.922238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.937748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.934412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.096941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.160538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.085670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.016431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.109866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.027301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.998433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.133765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.147689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.196907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.142714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.316521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.370177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.524846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.430770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.568744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.501153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.448303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.357431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.355850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.195132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.282799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.263624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.327227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.350943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.309260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.378953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.472946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.553685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.702246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.935293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.168207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.092284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.276362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.293537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.705199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.632067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.743735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.842845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.052824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.229315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.225485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.278820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.388720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.517699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.594688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.771922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.959260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.171156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.445924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.612756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.057379] LustreError: 163343:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6304.077941] LustreError: 163343:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6304.091384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.177428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.319407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.330461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.522388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.737782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.059322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.397307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.769163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.639961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.651674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.511636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.969230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.124942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.132383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.083799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.317166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.454230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.484342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.470766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.450783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.474249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.387534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.454660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.382510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.362544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.222153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.137946] LustreError: 163343:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6333.148579] LustreError: 163343:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6333.155970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.068850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.147754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.997914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.893765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.722853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.592195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.675767] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6339.681291] Lustre: Skipped 238 previous similar messages [ 6339.705977] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6339.722778] Lustre: Skipped 238 previous similar messages [ 6339.726757] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6339.733571] Lustre: Skipped 238 previous similar messages [ 6339.737694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.744370] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6339.747044] Lustre: Skipped 238 previous similar messages [ 6340.774770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.751405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.812795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.010962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.048738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.071892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.274653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.247409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.423799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.310129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.445762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.343682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.386417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.474956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.495947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.470713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.420828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.628306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.538363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.754435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.968855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.085522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.228819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.198338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.162372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.168778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.129766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.190391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.144687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.049215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.052538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.942937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.916525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.744329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.729217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.496497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.564325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.494921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.452693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.403714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.496215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.528283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.549704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.493621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.503283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.670475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.534765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.510964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.392911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.441174] LustreError: 163343:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6390.450733] LustreError: 163343:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6390.463761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.384687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.315936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.189899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.924279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.691799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.467709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.283882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.170320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.989098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.928295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.802981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.807577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.715047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.608778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.516520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.621338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.623894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.881983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.870739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.066675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.961779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.987277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.881373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.927708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.866428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.790118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.649534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.678669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.551375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.520425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.506837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.436632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.307080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.248561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.105845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.142604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.073417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.179538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.994328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.965748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.165178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.017217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.958552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.680697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.641195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.487245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.287943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.099320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.976833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.919684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.638623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.302276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.048531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.800459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.565857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.433958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.309388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.170685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.059197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.972702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.978138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.914046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.821110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.773866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.784985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.712967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.834542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.657394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.432213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.376536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.262191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.209760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.996793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.882339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.916084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.876748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.787131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.597521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.328874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.305674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.047445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.888033] LustreError: 163343:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6464.902027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.902269] LustreError: 163343:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6465.675430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.621955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.482060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.384883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.194754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.046036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.857828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.769828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.598251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.442581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.352886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.361453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.381241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.387046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.584608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.629988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.553804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.468519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.464360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.624805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.539320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.496634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6486.316539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.302150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.201861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.150645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.063436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.010343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.880859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.895184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.799697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.881175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.919990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.899702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.876280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.682460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.532114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.548284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.571095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.564859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.456644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.408765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.256360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.270522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.171696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.126903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.015603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.912516] LustreError: 163343:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6509.923443] LustreError: 163343:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6509.935876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.922406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.844777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.806067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.781877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.771180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.737114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.729213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.424460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.336819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.267474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6520.324382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.229878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.042230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.866540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.874626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.566471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.320858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.957260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6526.659303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.358241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.213576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6529.027585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6529.900660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6530.717259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.531362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.472925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.320434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6534.196771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.036063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.991879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.825975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.794609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6538.563394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.578171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.589307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.403512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.123875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.929493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.741937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.730834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6545.558504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.414733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.389345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.259822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.161486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.244734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.138336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6552.263671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.181955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.101552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.805414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.738822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.720665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.702278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.552629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.476351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.318866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.194210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.838485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.559818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.268159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.175490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.873718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.565928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.351162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.248629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.130045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.024870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.024189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.251384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.326936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.403551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.306922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.398295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.241036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.003884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.950624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.028684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.901702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.839780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.823861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.744117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.668191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.936084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.819759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6586.753453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.772471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.762662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.752329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.814879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.638199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6592.806056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6593.700941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.887934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.970983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.125560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.280383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.270739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.428081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.656961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.814778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.857220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.012114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.981163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6607.144732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.001232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.091854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.268886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.254454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.125954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.012365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.051713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.982358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6616.075884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6617.281059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.173883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6619.143560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.027575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.983507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6621.933959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.727190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6623.593604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.642844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6625.401770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6626.301473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.136945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.862788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.721332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.403511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6630.200965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6630.865635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6631.669356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6632.560228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6633.483879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6634.357781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6635.305222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6636.189972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6637.081221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6637.939049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6638.735673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.518305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6640.291463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6641.066302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6641.745830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6642.686882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6643.531936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6644.384588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6645.231700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6646.266832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.172335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6648.318196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6649.423605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.715974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6651.877702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6652.873967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6653.904940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.993891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6656.066624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6656.857309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6657.877228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6658.896723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.817267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6660.784389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.699481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6662.750437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6663.659896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6664.794423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6665.810705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6666.674270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6667.687298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6668.700586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6669.633797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6670.469722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6671.286211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6672.217782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6673.251468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6674.251055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6675.265898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6676.278860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6677.401149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6678.340484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6679.468706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6680.508955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6681.595874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6682.735434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6683.634288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6684.556406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6685.537593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6686.559267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6687.578926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6688.472214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6689.357685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6690.341956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6691.126592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6692.121902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6693.095123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6694.091084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6695.191982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6696.174442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6697.304900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6698.242899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6699.212598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6700.108721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6701.268949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6702.422352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6703.261903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6704.366983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6705.299494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6706.374115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6707.484250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6708.566520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6709.529715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6710.502597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6711.478274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6712.607986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6713.608318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6714.651367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6715.706756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6716.938948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6717.787555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6718.651536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6719.608063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6720.899382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6721.937280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6723.037184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6724.236876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6725.438386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6726.605986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6727.641340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6728.574714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6729.380727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6730.372229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6731.334801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6732.479775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6733.405441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6734.491244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6735.497920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6736.438674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6737.640590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6738.857090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6739.736556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6740.834722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6741.794030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6742.994057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6743.945494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6744.961628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6745.886919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6746.891127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6748.071216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6749.180663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6750.095694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6751.020614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6752.148385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6753.087669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6754.232609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6755.062962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6756.003180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6756.860809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6757.822193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6758.687298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6759.616620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6760.515920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6761.599596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6762.607048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6763.649502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6764.674986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6765.824066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6766.942722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6768.028720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6768.944474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6770.012824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6770.939725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6772.082784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6773.054400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6774.130696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6774.941201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6776.156599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6777.460669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6778.513319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6779.549353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6780.396110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6781.406179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6782.260621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6783.210442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6784.349601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6785.558421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6786.631833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6787.637816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6788.578503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6789.621299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6790.782761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6791.614495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6792.471658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6793.386451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6794.437586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6795.443924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6796.600399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6797.498988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6798.594856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6799.537455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6800.683418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6801.753719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6802.623232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6803.545277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6804.426321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6805.481434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6806.493623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6807.550735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6808.663764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6809.630310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6810.641783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6811.433329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6812.383138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6813.235484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6814.252468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6815.256410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6816.293913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6817.450842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6818.408587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6819.136496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6820.135384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6821.147970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6822.279904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6823.280504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6824.390560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6825.344392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6826.542488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6827.759372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6828.954240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6830.127153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6831.285961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6832.411870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6833.359495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6834.448375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6835.396781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6836.574662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6837.804614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6838.948848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6840.036893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6841.088863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6842.135415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6843.058329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6843.966775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6844.930833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6845.981605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6847.069203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6848.038922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6849.092208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6850.086716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6851.195976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6852.093834] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6852.096893] Lustre: Skipped 530 previous similar messages [ 6852.109289] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6852.114602] Lustre: Skipped 530 previous similar messages [ 6852.117641] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6852.120419] Lustre: Skipped 530 previous similar messages [ 6852.123273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6852.129158] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6852.131856] Lustre: Skipped 530 previous similar messages [ 6853.167371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6854.065899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6854.983529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6855.770605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6856.841710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6857.957143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6858.931546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6860.089162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7232.287371] LustreError: 166437:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7232.296820] LustreError: 163343:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7582.178597] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7582.180350] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7582.192931] Lustre: Skipped 9 previous similar messages [ 7587.300888] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7587.311176] Lustre: Skipped 2 previous similar messages [ 7587.667568] LustreError: 217269:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7587.815475] Lustre: server umount lustre-OST0000 complete [ 7589.866578] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7589.873498] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7592.417422] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7597.542321] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7597.546915] Lustre: Skipped 2 previous similar messages [ 7604.192363] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7604.266325] LustreError: 217471:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 7604.269280] LustreError: 217471:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7604.410672] Lustre: server umount lustre-MDT0000 complete [ 7606.351849] LustreError: 163304:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769080617 with bad export cookie 7422813629930012438 [ 7606.356978] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7606.357410] LustreError: 163304:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7606.435926] LustreError: 217673:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7606.438916] LustreError: 217673:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7606.517754] Lustre: server umount lustre-MDT0001 complete [ 7611.726525] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 7612.984489] Key type lgssc unregistered [ 7613.150256] LNet: 218205:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7613.153198] LNetError: 218205:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7613.162415] LNet: Removed LNI 192.168.202.152@tcp [ 7613.498783] Key type .llcrypt unregistered [ 7613.500345] Key type ._llcrypt unregistered [ 7625.718197] Key type ._llcrypt registered [ 7625.719812] Key type .llcrypt registered [ 7625.772108] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 7626.174093] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7626.226887] alg: No test for adler32 (adler32-zlib) [ 7627.133241] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 7627.241928] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 7628.848225] Key type lgssc registered [ 7629.449791] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7633.498282] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7633.506905] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7634.709270] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7636.630352] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 7641.090389] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7641.291254] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7643.306933] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 7646.183316] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7648.110839] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7651.886502] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7652.144205] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7655.497411] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 7658.419557] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7659.316207] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 7671.959113] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7672.991414] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7673.001134] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7673.006683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7673.013118] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7673.013145] LustreError: 219599:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7676.261315] Lustre: lustre-OST0000: Not available for connect from 192.168.202.52@tcp (stopping) [ 7676.901199] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7676.914290] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7676.914422] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7676.926094] Lustre: Skipped 1 previous similar message [ 7680.247762] LustreError: 221998:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7680.335937] Lustre: server umount lustre-OST0000 complete [ 7693.024875] LustreError: 219599:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7710.589521] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7710.750148] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7713.613340] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 7716.492318] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7716.904716] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7716.914714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7716.919867] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7716.925218] Lustre: Skipped 1 previous similar message [ 7718.382137] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 1 sec [ 7720.304741] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9c0e05afe000.ost_server_uuid 50 [ 7721.178924] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9c0e05afe000.ost_server_uuid in FULL state after 0 sec [ 7727.078837] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7727.082452] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7727.087516] Lustre: Skipped 2 previous similar messages [ 7727.100180] Lustre: Skipped 1 previous similar message [ 7729.265198] LustreError: 223317:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7729.272056] LustreError: 223317:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7729.323667] Lustre: server umount lustre-OST0000 complete [ 7732.193439] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7732.194472] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7732.197892] Lustre: Skipped 1 previous similar message [ 7737.158752] LustreError: 223518:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 7737.162644] LustreError: 223518:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7737.203824] Lustre: server umount lustre-MDT0000 complete [ 7737.314586] LustreError: 219579:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7737.324018] LustreError: 219579:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7739.344088] LustreError: 219561:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769080750 with bad export cookie 7828879799305978859 [ 7739.347232] LustreError: 219561:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7739.347912] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7739.422796] LustreError: 223720:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7739.425097] LustreError: 223720:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7739.477097] Lustre: server umount lustre-MDT0001 complete [ 7744.652980] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 7745.913608] Key type lgssc unregistered [ 7746.066655] LNet: 224251:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7746.070967] LNetError: 224251:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7746.080523] LNet: Removed LNI 192.168.202.152@tcp [ 7746.485352] Key type .llcrypt unregistered [ 7746.487402] Key type ._llcrypt unregistered [ 7761.246215] Key type ._llcrypt registered [ 7761.247203] Key type .llcrypt registered [ 7761.294409] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 7761.663729] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7761.671795] alg: No test for adler32 (adler32-zlib) [ 7762.595216] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 7762.714392] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 7764.304309] Key type lgssc registered [ 7764.803247] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7768.915342] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7768.922879] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7770.120924] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7771.732234] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 7775.407810] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7775.563689] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7777.188522] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 7778.729798] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7782.193481] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7782.367971] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7784.782801] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 7788.597910] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7788.631271] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7788.716332] Lustre: lustre-OST0001: new disk, initializing [ 7788.718436] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7788.773384] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7791.341675] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 7791.827982] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7791.832965] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7791.879548] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7794.927766] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 7797.496674] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7803.504487] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7816.673346] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7816.679699] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7816.687708] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7820.769892] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7820.770617] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7820.778725] Lustre: Skipped 1 previous similar message [ 7822.666600] LustreError: 229291:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 7822.710357] Lustre: server umount lustre-MDT0000 complete [ 7824.289215] LustreError: 225918:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769080835 with bad export cookie 8255280927713637362 [ 7824.290580] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7824.293958] LustreError: 225918:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7824.298681] LustreError: 226658:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 7824.379156] LustreError: 229493:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 7824.382313] LustreError: 229493:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 7824.458843] Lustre: server umount lustre-MDT0001 complete [ 7836.472079] LustreError: 229694:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7836.474098] LustreError: 229694:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 7836.495958] Lustre: server umount lustre-OST0000 complete [ 7847.284335] LustreError: 229895:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 7847.286926] LustreError: 229895:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7847.342556] Lustre: server umount lustre-OST0001 complete [ 7848.594665] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 7848.627524] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7849.049968] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 7849.081302] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7849.513186] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 7849.549509] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7849.969718] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 7850.008464] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7855.583781] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 7859.424234] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7859.634367] LustreError: 231303:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7859.641290] LustreError: 231303:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7859.669189] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7861.112880] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 7864.588531] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7866.430707] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 7867.737038] Lustre: 232414:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7870.759206] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7870.989148] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7870.991968] Lustre: Skipped 1 previous similar message [ 7873.432731] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 7875.110745] LustreError: 232769:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7875.116514] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 7877.076310] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7879.217498] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 7879.478629] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 7884.334306] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49761) [ 7888.811306] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7890.722107] Lustre: 234260:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7904.740355] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7904.744475] Lustre: Skipped 4 previous similar messages [ 7904.745478] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7904.750170] Lustre: Skipped 6 previous similar messages [ 7907.023248] LustreError: 234570:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 7907.025829] LustreError: 234570:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7907.062091] Lustre: server umount mylustre-MDT0000 complete [ 7908.644459] LustreError: 234463:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769080919 with bad export cookie 8255280927713640218 [ 7908.647728] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7908.650649] LustreError: 234463:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7908.773123] Lustre: server umount mylustre-MDT0001 complete [ 7920.575066] LustreError: 234973:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 7920.579381] LustreError: 234973:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 7920.601141] Lustre: server umount mylustre-OST0000 complete [ 7932.583227] Lustre: server umount mylustre-OST0001 complete [ 7933.952656] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 7933.982636] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7934.446548] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 7934.480693] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7934.933859] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 7934.976221] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7935.457210] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 7935.497362] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7941.459082] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 7945.449780] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7945.640474] LustreError: 236583:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7945.667541] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7945.670177] Lustre: Skipped 1 previous similar message [ 7947.407795] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 7950.817659] LustreError: 236583:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7950.898976] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7952.634484] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 7953.790378] Lustre: 237695:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7956.366382] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7958.906606] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 7960.613463] LustreError: 238569:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7960.621823] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 7962.739512] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7964.415094] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:65) [ 7965.182042] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 7971.565746] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49793) [ 7974.366561] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7975.779995] Lustre: 239537:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7987.171589] Lustre: tfs-MDT0000-lwp-OST0000: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7987.173533] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7987.181299] Lustre: Skipped 2 previous similar messages [ 7987.182154] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7987.184055] Lustre: Skipped 2 previous similar messages [ 7989.124864] LustreError: 239789:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 7989.127230] LustreError: 239789:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 7989.158835] Lustre: server umount tfs-MDT0000 complete [ 7990.676107] LustreError: 236559:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769081001 with bad export cookie 8255280927713642395 [ 7990.677024] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7990.680328] LustreError: 236559:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7990.823477] Lustre: server umount tfs-MDT0001 complete [ 8002.818840] Lustre: server umount tfs-OST0000 complete [ 8014.628181] Lustre: server umount tfs-OST0001 complete [ 8015.797849] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 8015.822767] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8016.189043] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 8016.216032] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8016.609057] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 8016.629363] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8017.002750] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 8017.030559] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8021.908262] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 8025.611935] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8025.772927] LustreError: 241801:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8025.796138] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8025.798489] Lustre: Skipped 3 previous similar messages [ 8027.229638] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8030.364341] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8032.038587] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8033.482259] Lustre: 242910:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8035.745407] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8037.801840] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8040.866263] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8041.963133] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:97) [ 8041.963226] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 8043.223498] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8051.253840] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49825) [ 8059.189819] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8062.038735] Lustre: 244779:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8072.546196] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 06:24:41 (1769081081) [ 8087.013325] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8087.014135] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8087.016160] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8087.023108] Lustre: Skipped 2 previous similar messages [ 8092.233446] LustreError: 245146:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 8092.235806] LustreError: 245146:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 8092.278606] Lustre: server umount lustre-MDT0000 complete [ 8094.151691] LustreError: 241777:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769081104 with bad export cookie 8255280927713644313 [ 8094.155047] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8094.159152] LustreError: 241777:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8097.252774] LustreError: 244796:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8097.253576] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8097.262220] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8097.262230] Lustre: Skipped 7 previous similar messages [ 8097.265879] LustreError: 244796:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8097.271417] Lustre: Skipped 1 previous similar message [ 8100.370303] Lustre: server umount lustre-MDT0001 complete [ 8108.228845] Lustre: server umount lustre-OST0000 complete [ 8115.690212] Lustre: server umount lustre-OST0001 complete [ 8118.238607] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_hostid [ 8122.758474] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 8127.807085] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8131.338390] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8134.549512] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8137.668396] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8141.230931] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8141.254365] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 8141.254365] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8141.254365] [ 8141.262989] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8141.389659] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8141.407720] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8141.447237] Lustre: lustre-MDT0000: new disk, initializing [ 8141.490561] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8141.493863] Lustre: Skipped 3 previous similar messages [ 8141.503686] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8143.322926] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8149.211669] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8149.246870] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8149.294774] Lustre: 247932:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8149.333600] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8149.335951] Lustre: Skipped 1 previous similar message [ 8149.375443] Lustre: lustre-MDT0001: new disk, initializing [ 8149.449152] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8149.456040] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8151.084607] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8156.090663] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8156.113268] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8156.227985] Lustre: lustre-OST0000: new disk, initializing [ 8156.230216] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8157.632042] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8157.635421] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8157.680812] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8158.645100] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8163.655896] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8163.690342] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8163.773691] Lustre: lustre-OST0001: new disk, initializing [ 8163.776615] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8165.516632] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8165.520292] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8165.569113] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8166.184893] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8171.604768] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 06:26:21 (1769081181) [ 8176.515465] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 06:26:26 (1769081186) [ 8188.385902] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8188.386104] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8188.388511] LustreError: Skipped 1 previous similar message [ 8188.396040] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8188.400303] Lustre: Skipped 4 previous similar messages [ 8194.367148] LustreError: 251355:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 8194.370842] LustreError: 251355:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 8194.397161] Lustre: server umount lustre-OST0000 complete [ 8196.581932] LustreError: 249676:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8196.596589] LustreError: 249676:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8202.384414] Lustre: server umount lustre-MDT0000 complete [ 8204.214096] LustreError: 248715:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769081215 with bad export cookie 8255280927713646091 [ 8204.218873] LustreError: 248715:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8204.219167] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8210.465197] Lustre: server umount lustre-MDT0001 complete [ 8214.954619] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 8226.476850] Lustre: server umount lustre-OST0001 complete [ 8229.628425] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_hostid [ 8237.593677] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 8242.192594] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8245.164222] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8247.836843] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8250.550681] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8253.402194] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8253.427600] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8253.535349] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8253.553266] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8253.591831] Lustre: lustre-MDT0000: new disk, initializing [ 8253.648843] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8255.294456] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8260.319194] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8260.340659] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8260.383958] Lustre: 254818:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8260.504479] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8262.060126] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8265.441394] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8266.819107] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8269.338339] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8269.376060] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8269.513714] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8269.516716] Lustre: Skipped 5 previous similar messages [ 8271.420391] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8271.424414] Lustre: Skipped 1 previous similar message [ 8271.426436] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8271.466468] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8271.628268] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8275.039699] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8281.572945] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8281.574151] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8281.574341] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8281.583315] Lustre: Skipped 6 previous similar messages [ 8281.586742] Lustre: Skipped 10 previous similar messages [ 8283.802157] Lustre: server umount lustre-OST0000 complete [ 8291.559403] Lustre: server umount lustre-MDT0000 complete [ 8291.811801] LustreError: 254828:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8291.820371] LustreError: 254828:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 8293.179323] LustreError: 255599:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769081304 with bad export cookie 8255280927713647561 [ 8293.181699] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8293.183912] LustreError: 255599:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8293.334100] Lustre: server umount lustre-MDT0001 complete [ 8297.410686] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 8298.576229] Key type lgssc unregistered [ 8298.726698] LNet: 258193:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8298.731213] LNetError: 258193:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8298.744301] LNet: Removed LNI 192.168.202.152@tcp [ 8299.120047] Key type .llcrypt unregistered [ 8299.125029] Key type ._llcrypt unregistered [ 8312.224591] Key type ._llcrypt registered [ 8312.226026] Key type .llcrypt registered [ 8312.280857] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 8312.635651] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8312.645509] alg: No test for adler32 (adler32-zlib) [ 8313.553642] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 8313.674458] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 8315.272204] Key type lgssc registered [ 8315.724352] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8319.053165] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8319.061236] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8320.255342] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8321.716092] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8325.257225] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8325.455768] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8327.124432] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8329.510433] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8331.067793] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8333.603739] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8333.745175] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8335.849728] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8337.978148] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8338.347561] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8340.962787] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8340.963083] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8340.968433] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8340.971147] LustreError: Skipped 1 previous similar message [ 8345.067966] LustreError: 261908:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 8345.103207] Lustre: server umount lustre-OST0000 complete [ 8351.202284] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8351.202337] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8351.203951] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8351.224184] Lustre: Skipped 1 previous similar message [ 8352.807113] LustreError: 262113:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 8352.811529] LustreError: 262113:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8352.849321] Lustre: server umount lustre-MDT0000 complete [ 8354.975040] LustreError: 259594:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769081365 with bad export cookie 6618680297952595327 [ 8354.976724] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8354.980702] LustreError: 259594:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8354.981162] LustreError: 260328:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0001: can't precreate: rc = -5 [ 8354.992718] LustreError: 260328:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0001: cannot precreate objects: rc = -5 [ 8355.058152] LustreError: 262315:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 8355.060246] LustreError: 262315:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8355.148721] Lustre: server umount lustre-MDT0001 complete [ 8359.617909] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 8360.942932] Key type lgssc unregistered [ 8361.092862] LNet: 262845:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8361.097085] LNetError: 262845:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8361.103493] LNet: Removed LNI 192.168.202.152@tcp [ 8361.453444] Key type .llcrypt unregistered [ 8361.454604] Key type ._llcrypt unregistered [ 8368.838265] Key type ._llcrypt registered [ 8368.839774] Key type .llcrypt registered [ 8368.882730] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 8369.263074] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8369.316464] alg: No test for adler32 (adler32-zlib) [ 8370.185753] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 8370.274588] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 8371.864177] Key type lgssc registered [ 8372.294549] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8375.718227] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8375.725183] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8391.200352] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8396.320204] LustreError: 263996:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.152@tcp: failed processing log, type 4: rc = -110 [ 8422.944179] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8425.101567] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8427.443496] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8428.362652] LustreError: 264660:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 8428.407275] Lustre: server umount lustre-OST0000 complete [ 8434.449204] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 8435.597440] Key type lgssc unregistered [ 8435.742404] LNet: 265495:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8435.745995] LNetError: 265495:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8435.756419] LNet: Removed LNI 192.168.202.152@tcp [ 8436.034626] Key type .llcrypt unregistered [ 8436.035691] Key type ._llcrypt unregistered [ 8449.837488] Key type ._llcrypt registered [ 8449.838452] Key type .llcrypt registered [ 8449.877350] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 8450.170651] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8450.200063] alg: No test for adler32 (adler32-zlib) [ 8451.048603] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 8451.127798] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 8452.704135] Key type lgssc registered [ 8453.092094] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8457.656689] loop0: detected capacity change from 0 to 204800000 [ 8457.719709] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8457.722949] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8457.844182] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8457.866306] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8458.217251] loop0: detected capacity change from 0 to 204800000 [ 8458.275568] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8458.279363] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8458.401220] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8458.424468] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8458.783185] loop0: detected capacity change from 0 to 204800000 [ 8458.893028] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8458.896658] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8458.940359] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8458.987130] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8459.347436] loop0: detected capacity change from 0 to 204800000 [ 8459.408498] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8459.492289] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8459.820794] loop0: detected capacity change from 0 to 204800000 [ 8459.833179] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8461.188533] loop0: detected capacity change from 0 to 204800000 [ 8461.201607] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8462.664663] loop0: detected capacity change from 0 to 204800000 [ 8462.675211] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8464.094079] loop0: detected capacity change from 0 to 204800000 [ 8464.109397] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8465.476901] loop0: detected capacity change from 0 to 204800000 [ 8465.511372] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8465.545648] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8465.552189] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8465.579185] Lustre: 268871:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8465.582845] Lustre: 268871:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8465.586861] Lustre: 268871:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8465.591928] Lustre: 268871:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8465.596689] Lustre: 268871:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8465.600987] Lustre: 268871:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8465.605476] Lustre: 268871:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8465.609846] Lustre: 268871:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8465.614613] Lustre: 268871:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8465.619266] Lustre: 268871:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8465.622878] Lustre: 268871:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8465.626757] Lustre: 268871:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8465.630100] Lustre: 268871:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8465.633656] Lustre: 268871:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8468.348453] LustreError: 269137:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 8468.397063] Lustre: server umount MGS complete [ 8468.723536] loop0: detected capacity change from 0 to 204800000 [ 8468.760752] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8468.949189] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8468.968051] Lustre: lustre-MDT0000: Aborting client recovery [ 8469.299707] loop1: detected capacity change from 0 to 204800000 [ 8469.350055] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8469.393975] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8469.417099] Lustre: 269304:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8469.421231] Lustre: 269304:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8469.425411] Lustre: 269304:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8469.429609] Lustre: 269304:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8469.433719] Lustre: 269304:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8469.437790] Lustre: 269304:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8469.441617] Lustre: 269304:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8469.448286] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8469.529125] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8469.558646] Lustre: lustre-MDT0001: Aborting client recovery [ 8469.914745] loop2: detected capacity change from 0 to 204800000 [ 8469.977051] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8470.007878] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8470.023893] Lustre: 269391:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8470.027959] Lustre: 269391:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8470.031922] Lustre: 269391:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8470.036046] Lustre: 269391:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8470.040279] Lustre: 269391:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8470.044578] Lustre: 269391:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8470.048416] Lustre: 269391:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8470.099118] Lustre: Found index 0 for lustre-OST0000, updating log [ 8470.138033] Lustre: lustre-OST0000: Aborting recovery [ 8470.498056] loop3: detected capacity change from 0 to 204800000 [ 8470.565973] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8470.591315] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8470.611785] Lustre: 269480:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8470.616946] Lustre: 269480:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8470.621932] Lustre: 269480:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8470.626886] Lustre: 269480:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8470.630970] Lustre: 269480:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8470.636093] Lustre: 269480:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8470.640335] Lustre: 269480:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8470.684515] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8470.687215] Lustre: Skipped 1 previous similar message [ 8470.694980] Lustre: lustre-OST0001: Aborting recovery [ 8476.137695] LustreError: 269247:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 8478.179639] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8478.179724] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8478.189484] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8478.192695] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8479.203070] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8479.203214] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8481.606203] LustreError: 269215:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 324us: rc = -2 [ 8494.561710] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8494.562886] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8494.565671] Lustre: Skipped 2 previous similar messages [ 8494.568452] Lustre: Skipped 3 previous similar messages [ 8498.814950] LustreError: 269836:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 8498.817582] LustreError: 269836:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 8498.859822] Lustre: server umount lustre-MDT0000 complete [ 8499.194592] LustreError: 269199:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769081510 with bad export cookie 9850828712866112509 [ 8499.195083] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8499.199558] LustreError: 269199:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8499.680479] LustreError: 269213:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8499.680743] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8499.681261] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8499.686595] LustreError: 269213:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8499.691055] Lustre: Skipped 2 previous similar messages [ 8504.800737] LustreError: 269889:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8504.800754] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8504.805531] LustreError: 269889:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8504.807718] Lustre: Skipped 2 previous similar messages [ 8505.347375] LustreError: 269888:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 8505.349573] LustreError: 269888:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8505.410463] Lustre: server umount lustre-MDT0001 complete [ 8520.160127] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8520.177619] LustreError: 269940:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 8520.179570] LustreError: 269940:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8520.209782] Lustre: server umount lustre-OST0000 complete [ 8526.638105] LustreError: 269992:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 8526.641103] LustreError: 269992:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8526.699561] Lustre: server umount lustre-OST0001 complete [ 8529.401119] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 06:32:19 (1769081539) [ 8538.627258] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_hostid [ 8541.589983] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 8544.875309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8546.989628] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8549.117316] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8551.317480] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8553.546812] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8553.567783] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8553.648251] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8553.661787] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8553.696699] Lustre: lustre-MDT0000: new disk, initializing [ 8553.721721] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8553.728684] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8555.044569] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8559.232996] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8559.258535] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8559.295476] Lustre: 273536:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8559.322622] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8559.325174] Lustre: Skipped 1 previous similar message [ 8559.362140] Lustre: lustre-MDT0001: new disk, initializing [ 8559.401517] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8559.413293] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8559.416888] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8560.793365] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8563.956416] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8565.136595] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8567.344714] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8567.365635] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8567.453461] Lustre: lustre-OST0000: new disk, initializing [ 8567.456801] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8567.483519] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8569.332423] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8570.551056] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8570.555174] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8570.586772] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8572.525321] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8574.916650] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8581.230818] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8589.498741] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8596.003506] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8637.409871] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8637.411237] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8637.412093] LustreError: Skipped 1 previous similar message [ 8637.416583] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8637.419122] Lustre: Skipped 1 previous similar message [ 8641.710718] LustreError: 277244:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 8641.712938] LustreError: 277244:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8641.748251] Lustre: server umount lustre-OST0000 complete [ 8646.624768] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8646.627223] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8646.631351] Lustre: Skipped 1 previous similar message [ 8646.633139] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8657.376090] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8657.431529] Lustre: server umount lustre-MDT0000 complete [ 8657.889379] LustreError: 273546:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8657.895457] LustreError: 273546:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8658.586122] LustreError: 273528:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769081669 with bad export cookie 9850828712866115211 [ 8658.588444] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8658.589741] LustreError: 273528:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8658.655502] LustreError: 277649:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 8658.657427] LustreError: 277649:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 8658.706844] Lustre: server umount lustre-MDT0001 complete [ 8661.424611] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8662.595791] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8664.068018] Lustre: server umount MGS complete [ 8667.499918] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8667.625911] LustreError: 279210:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8667.631960] LustreError: 279222:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8667.649963] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8668.803786] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8671.469063] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8672.760188] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8674.399456] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8675.483435] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8677.609775] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8679.362844] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8680.955626] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8702.434908] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8702.436874] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8702.442058] Lustre: Skipped 1 previous similar message [ 8702.444474] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8702.447012] Lustre: Skipped 5 previous similar messages [ 8705.712190] LustreError: 281870:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 8705.714761] LustreError: 281870:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 8705.748562] Lustre: server umount lustre-OST0000 complete [ 8707.555024] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8721.376145] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8721.434140] Lustre: server umount lustre-MDT0000 complete [ 8722.912937] LustreError: 279210:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8722.918230] LustreError: 279210:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8722.960259] LustreError: 279192:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769081733 with bad export cookie 9850828712866117052 [ 8722.963453] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8722.964496] LustreError: 279192:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8723.086596] Lustre: server umount lustre-MDT0001 complete [ 8726.697225] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 8727.788428] Key type lgssc unregistered [ 8727.918385] LNet: 282807:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8727.922328] LNetError: 282807:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8727.931444] LNet: Removed LNI 192.168.202.152@tcp [ 8728.236854] Key type .llcrypt unregistered [ 8728.237916] Key type ._llcrypt unregistered [ 8739.584457] Key type ._llcrypt registered [ 8739.585842] Key type .llcrypt registered [ 8739.620776] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_hostid [ 8745.768879] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 8746.108461] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8746.137579] alg: No test for adler32 (adler32-zlib) [ 8746.977284] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 8747.050862] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 8748.632137] Key type lgssc registered [ 8748.968180] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8753.263352] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8755.366678] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8757.362940] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8759.430697] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8761.607554] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8761.627160] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8761.631648] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8762.707102] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8762.717396] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8762.752384] Lustre: lustre-MDT0000: new disk, initializing [ 8762.773693] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8762.779411] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8763.971019] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8768.069041] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8768.099487] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8768.140776] Lustre: 286709:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8768.166623] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8768.167981] Lustre: Skipped 1 previous similar message [ 8768.200065] Lustre: lustre-MDT0001: new disk, initializing [ 8768.232926] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8768.246170] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8768.249122] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8769.595913] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8773.011368] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8774.495636] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8777.512314] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8777.539265] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8777.647094] Lustre: lustre-OST0000: new disk, initializing [ 8777.649955] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8777.689470] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8780.294723] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8784.116601] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8784.909095] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8784.918705] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8784.959431] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8788.795617] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8792.608587] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8799.830463] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8851.426641] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8851.427316] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8851.430764] LustreError: Skipped 1 previous similar message [ 8851.435482] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8851.439552] Lustre: Skipped 1 previous similar message [ 8853.743069] LustreError: 290293:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 8853.782392] Lustre: server umount lustre-OST0000 complete [ 8855.521147] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8855.525070] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8855.529915] Lustre: Skipped 1 previous similar message [ 8855.532708] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8856.545602] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8856.545922] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8861.256071] LustreError: 290494:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 8861.258680] LustreError: 290494:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8861.291300] Lustre: server umount lustre-MDT0000 complete [ 8862.726390] LustreError: 286701:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769081873 with bad export cookie 14313663862765635077 [ 8862.727746] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8862.738843] LustreError: 286701:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8862.799813] LustreError: 290695:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 8862.802548] LustreError: 290695:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8862.870907] Lustre: server umount lustre-MDT0001 complete [ 8866.169180] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8867.682615] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8869.435168] LustreError: 291635:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 8869.437495] LustreError: 291635:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8869.461870] Lustre: server umount MGS complete [ 8873.626314] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8873.780032] LustreError: 292258:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8873.787898] LustreError: 292258:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8873.787979] LustreError: 292270:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8873.819960] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8875.207387] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8878.209091] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8878.327549] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8879.673331] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8881.621473] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8882.999313] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8885.351440] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8887.371110] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8889.312428] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8919.524328] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8919.524481] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8919.525877] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8919.525883] Lustre: Skipped 1 previous similar message [ 8919.526288] LustreError: Skipped 1 previous similar message [ 8919.529280] Lustre: Skipped 1 previous similar message [ 8924.642089] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8924.644426] Lustre: Skipped 2 previous similar messages [ 8924.846455] LustreError: 294937:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 8924.848526] LustreError: 294937:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 8924.889985] Lustre: server umount lustre-OST0000 complete [ 8929.761485] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8929.761935] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8932.313497] Lustre: server umount lustre-MDT0000 complete [ 8933.574043] LustreError: 293878:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769081944 with bad export cookie 14313663862765636918 [ 8933.577269] LustreError: 293878:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8933.577420] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8933.639074] LustreError: 295341:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 8933.640640] LustreError: 295341:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 8933.690433] Lustre: server umount lustre-MDT0001 complete [ 8937.226320] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 8938.370805] Key type lgssc unregistered [ 8938.509435] LNet: 295872:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8938.512243] LNetError: 295872:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8938.521334] LNet: Removed LNI 192.168.202.152@tcp [ 8938.819742] Key type .llcrypt unregistered [ 8938.821127] Key type ._llcrypt unregistered [ 8946.005182] Key type ._llcrypt registered [ 8946.006017] Key type .llcrypt registered [ 8946.043814] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_hostid [ 8952.100458] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 8952.577835] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8952.599102] alg: No test for adler32 (adler32-zlib) [ 8953.465752] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 8953.555244] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 8955.144145] Key type lgssc registered [ 8955.533859] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8958.553050] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8960.656524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8963.826600] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 8966.785960] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8969.247319] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8969.274284] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8969.280038] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8970.377555] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8970.390174] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8970.427628] Lustre: lustre-MDT0000: new disk, initializing [ 8970.452630] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8970.459451] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8971.876669] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 8976.347848] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8976.376028] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8976.471253] Lustre: lustre-OST0000: new disk, initializing [ 8976.473198] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8976.476041] Lustre: Skipped 1 previous similar message [ 8976.497947] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8977.220494] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 8977.224799] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 8977.235984] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 8978.486929] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9182.177893] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9182.183178] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9185.860574] LustreError: 300788:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 9185.980833] Lustre: server umount lustre-MDT0000 complete [ 9187.331195] LustreError: 299033:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769082198 with bad export cookie 953606283890418865 [ 9187.333483] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9187.335086] LustreError: 299033:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9187.369302] LustreError: 300990:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 9187.372157] LustreError: 300990:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9187.423363] Lustre: server umount lustre-OST0000 complete [ 9196.351500] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_hostid [ 9199.690097] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 9203.379983] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9205.681522] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9207.885561] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9210.104939] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9212.511148] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9212.534113] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9212.624049] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9212.635602] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9212.671855] Lustre: lustre-MDT0000: new disk, initializing [ 9212.700578] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9212.709389] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9214.088300] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9218.291160] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9218.308758] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9218.344312] Lustre: 304027:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9218.370871] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9218.373132] Lustre: Skipped 1 previous similar message [ 9218.415773] Lustre: lustre-MDT0001: new disk, initializing [ 9218.452240] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9218.469134] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9218.472973] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9219.811158] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9222.939245] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9224.160213] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9226.451475] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9226.483903] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9226.588560] Lustre: lustre-OST0000: new disk, initializing [ 9226.590783] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9226.622097] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9228.607728] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9230.329171] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9230.332445] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9230.360834] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9231.717701] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9233.168766] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9233.252142] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9234.694348] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9234.762315] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9235.426546] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9235.427787] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9235.431144] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9235.434585] Lustre: Skipped 1 previous similar message [ 9235.437783] Lustre: Skipped 1 previous similar message [ 9240.549074] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9240.551582] Lustre: Skipped 1 previous similar message [ 9241.519105] LustreError: 306965:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 9241.521342] LustreError: 306965:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9241.561624] Lustre: server umount lustre-OST0000 complete [ 9244.128888] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9244.131544] LustreError: Skipped 1 previous similar message [ 9244.133144] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9244.137943] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9249.020632] LustreError: 307166:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 9249.022178] LustreError: 307166:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9249.049701] Lustre: server umount lustre-MDT0000 complete [ 9250.225411] LustreError: 305772:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769082261 with bad export cookie 953606283893359649 [ 9250.228488] LustreError: 305772:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9250.229273] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9250.346192] Lustre: server umount lustre-MDT0001 complete [ 9251.448328] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 9252.005097] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 06:44:21 (1769082261) [ 9254.130484] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9254.322481] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9255.523019] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9258.295711] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9259.732707] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9261.463434] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9262.634883] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9264.995288] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9267.002091] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9268.760200] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9270.963061] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9270.985538] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9271.035565] Lustre: lustre-OST0001: new disk, initializing [ 9271.037609] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9271.065965] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9271.068260] Lustre: Skipped 2 previous similar messages [ 9272.911277] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9280.514804] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9280.518596] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9280.533465] LustreError: 311008:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9280.536094] LustreError: 311008:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9281.571723] LustreError: 311008:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9281.574709] LustreError: 311008:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9281.674799] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9281.755572] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9282.595052] LustreError: 311012:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9282.597727] LustreError: 311012:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9282.946989] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9c0e07846000.ost_server_uuid 50 [ 9283.448181] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9c0e07846000.ost_server_uuid in FULL state after 0 sec [ 9284.643203] LustreError: 311012:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9284.646252] LustreError: 311012:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 9284.675549] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9c0e07846000.ost_server_uuid 50 [ 9285.190567] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9c0e07846000.ost_server_uuid in FULL state after 0 sec [ 9285.676971] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9300.829843] Lustre: lustre-OST0001: Not available for connect from 192.168.202.52@tcp (stopping) [ 9300.831913] Lustre: Skipped 2 previous similar messages [ 9300.964851] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9300.964901] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9300.966763] LustreError: Skipped 1 previous similar message [ 9300.969810] Lustre: Skipped 1 previous similar message [ 9311.064382] Lustre: lustre-OST0001: Not available for connect from 192.168.202.52@tcp (stopping) [ 9311.066803] Lustre: Skipped 5 previous similar messages [ 9311.200139] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9311.224484] LustreError: 311744:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 9311.225974] LustreError: 311744:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 9311.242491] Lustre: server umount lustre-OST0001 complete [ 9314.272795] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9314.274711] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9316.323729] LustreError: 311400:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9316.328248] LustreError: 311400:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9318.801162] Lustre: server umount lustre-OST0000 complete [ 9331.691180] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9331.692890] Lustre: Skipped 11 previous similar messages [ 9334.240078] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9334.291479] LustreError: 312148:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 9334.293433] LustreError: 312148:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 9334.320913] Lustre: server umount lustre-MDT0000 complete [ 9335.551067] LustreError: 307965:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769082346 with bad export cookie 953606283893360517 [ 9335.555979] LustreError: 307965:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9335.556489] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9335.689410] Lustre: server umount lustre-MDT0001 complete [ 9339.180306] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 9340.176175] Key type lgssc unregistered [ 9340.277436] LNet: 312881:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9340.279406] LNetError: 312881:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9340.290450] LNet: Removed LNI 192.168.202.152@tcp [ 9340.576208] Key type .llcrypt unregistered [ 9340.577539] Key type ._llcrypt unregistered [ 9349.091446] Key type ._llcrypt registered [ 9349.092384] Key type .llcrypt registered [ 9349.125838] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 9349.529951] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9349.549877] alg: No test for adler32 (adler32-zlib) [ 9350.389672] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 9350.463729] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 9352.040161] Key type lgssc registered [ 9352.389729] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9354.808291] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9354.814263] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9355.962487] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9357.193625] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9358.788517] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9360.954380] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9361.090163] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9362.329659] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9365.266296] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9365.365340] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9367.081800] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9368.806960] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9370.469358] LustreError: 315811:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9370.475165] LustreError: 315811:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9371.079933] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9371.154387] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9372.907034] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9374.605713] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 9381.034391] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 9381.034788] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 9382.522943] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 9382.599345] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9458.146594] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9458.149614] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9458.153872] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9458.155715] Lustre: Skipped 1 previous similar message [ 9463.267870] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9463.269247] Lustre: Skipped 1 previous similar message [ 9468.388907] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9468.390889] Lustre: Skipped 1 previous similar message [ 9471.968116] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9471.988086] LustreError: 317682:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 9472.003199] Lustre: server umount lustre-OST0001 complete [ 9473.506164] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9473.506614] LustreError: 315809:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9473.506634] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9473.506643] Lustre: Skipped 1 previous similar message [ 9473.508171] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9473.512011] LustreError: 315809:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9473.518498] Lustre: Skipped 1 previous similar message [ 9478.626193] LustreError: 316603:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9478.626283] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9478.630592] LustreError: 316603:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9478.633082] Lustre: Skipped 1 previous similar message [ 9479.533927] LustreError: 317885:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 9479.535652] LustreError: 317885:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9479.567510] Lustre: server umount lustre-OST0000 complete [ 9483.745726] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9483.748966] Lustre: Skipped 2 previous similar messages [ 9488.865129] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9488.867192] Lustre: Skipped 3 previous similar messages [ 9495.008132] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9495.051765] LustreError: 318087:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 9495.053759] LustreError: 318087:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9495.085085] Lustre: server umount lustre-MDT0000 complete [ 9496.330147] LustreError: 314238:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769082507 with bad export cookie 14835738454915862506 [ 9496.333465] LustreError: 314238:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9496.334416] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9496.456659] Lustre: server umount lustre-MDT0001 complete [ 9500.222902] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 9501.266227] Key type lgssc unregistered [ 9501.399432] LNet: 318820:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9501.401553] LNetError: 318820:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9501.411398] LNet: Removed LNI 192.168.202.152@tcp [ 9501.701264] Key type .llcrypt unregistered [ 9501.702590] Key type ._llcrypt unregistered [ 9519.009736] Key type ._llcrypt registered [ 9519.010992] Key type .llcrypt registered [ 9519.041674] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 9519.444390] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9519.450540] alg: No test for adler32 (adler32-zlib) [ 9520.286484] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 9520.359298] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 9521.936105] Key type lgssc registered [ 9522.265900] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9525.114892] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9525.120849] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9526.254217] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9527.401772] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9529.971588] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9530.087955] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9531.233801] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9532.107470] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 9534.199849] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9534.308193] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9535.999277] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9536.358046] LustreError: 322947:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9536.360931] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 9536.363264] LustreError: 322947:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9538.625339] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9538.694323] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9539.753665] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 9540.387836] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9544.293922] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9549.843129] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 9560.546646] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9560.548899] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9560.550080] Lustre: Skipped 2 previous similar messages [ 9560.551512] Lustre: Skipped 1 previous similar message [ 9564.414098] LustreError: 324474:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 9564.438978] Lustre: server umount lustre-MDT0000 complete [ 9565.604100] LustreError: 321047:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769082576 with bad export cookie 5185295073294487138 [ 9565.608086] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9565.608718] LustreError: 321047:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9565.665837] LustreError: 321780:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9565.667283] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9565.668162] LustreError: 324675:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 9565.668168] LustreError: 324675:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 9565.669231] LustreError: 321780:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9565.673201] Lustre: Skipped 1 previous similar message [ 9565.676092] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9565.680783] Lustre: Skipped 2 previous similar messages [ 9565.717622] Lustre: server umount lustre-MDT0001 complete [ 9572.975082] LustreError: 324877:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 9572.977820] LustreError: 324877:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 9572.997859] Lustre: server umount lustre-OST0000 complete [ 9574.299583] Lustre: server umount lustre-OST0001 complete [ 9579.892156] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 9583.516224] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9583.659274] LustreError: 326252:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9583.666850] LustreError: 326252:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9583.681490] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9585.033236] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9587.949390] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9589.296940] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9590.213860] Lustre: 327361:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 9592.320487] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9592.438980] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9592.441245] Lustre: Skipped 1 previous similar message [ 9593.442946] LustreError: 327716:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9593.446387] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 9594.178963] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9596.786982] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9597.866305] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 9598.614611] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9601.630889] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9602.825821] Lustre: 329208:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 9613.283703] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9613.284966] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9613.287571] Lustre: Skipped 2 previous similar messages [ 9613.292338] Lustre: Skipped 3 previous similar messages [ 9614.843325] LustreError: 329311:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 9614.844784] LustreError: 329311:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 9614.870222] Lustre: server umount lustre-MDT0000 complete [ 9616.026988] LustreError: 329210:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769082626 with bad export cookie 5185295073294488797 [ 9616.029986] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9616.030424] LustreError: 329210:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9616.143180] Lustre: server umount lustre-MDT0001 complete [ 9627.636929] LustreError: 329716:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 9627.638490] LustreError: 329716:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 9627.652831] Lustre: server umount lustre-OST0000 complete [ 9639.090509] LustreError: 329917:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 9639.092314] LustreError: 329917:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9639.132947] Lustre: server umount lustre-OST0001 complete [ 9640.384939] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 9640.934720] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 9641.541881] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 06:50:51 (1769082651) [ 9648.917113] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 9651.944395] loop0: detected capacity change from 0 to 1099511627776 [ 9652.068093] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 9652.070513] *** VALIDATE xfs *** [ 9652.075052] XFS (loop0): Mounting V5 Filesystem [ 9652.083033] XFS (loop0): Ending clean mount [ 9652.505957] loop1: detected capacity change from 0 to 18691697672192 [ 9666.413497] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9671.584617] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_hostid [ 9674.579091] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 9677.584458] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9679.589308] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9681.479683] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9683.524484] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9685.727048] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9685.750175] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9685.828063] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9685.839283] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9685.872512] Lustre: lustre-MDT0000: new disk, initializing [ 9685.894648] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9685.896320] Lustre: Skipped 1 previous similar message [ 9685.902450] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9687.159426] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9691.150508] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9691.174611] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9691.203937] Lustre: 334284:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9691.227596] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9691.229716] Lustre: Skipped 1 previous similar message [ 9691.263018] Lustre: lustre-MDT0001: new disk, initializing [ 9691.302702] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9691.306676] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9692.564375] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9695.544634] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9696.673328] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9698.757246] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9698.779441] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9698.860420] Lustre: lustre-OST0000: new disk, initializing [ 9698.862566] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9700.213479] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9700.217239] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9700.250065] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9700.611406] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9703.604989] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9704.946073] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9705.020793] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9706.301137] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9706.372094] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9710.562430] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9710.563070] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9710.564633] LustreError: Skipped 1 previous similar message [ 9710.568027] Lustre: Skipped 1 previous similar message [ 9710.570696] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9710.573422] Lustre: Skipped 1 previous similar message [ 9715.684574] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9715.686451] Lustre: Skipped 1 previous similar message [ 9721.312095] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9721.329153] LustreError: 337222:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 9721.331046] LustreError: 337222:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 9721.366460] Lustre: server umount lustre-OST0000 complete [ 9725.921579] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9725.922517] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9725.924945] Lustre: Skipped 1 previous similar message [ 9725.927111] Lustre: Skipped 2 previous similar messages [ 9728.663991] Lustre: server umount lustre-MDT0000 complete [ 9729.818398] LustreError: 334276:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769082740 with bad export cookie 5185295073294490351 [ 9729.821590] LustreError: 334276:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9729.822579] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9729.942897] Lustre: server umount lustre-MDT0001 complete [ 9732.449411] XFS (loop0): Unmounting Filesystem [ 9733.862885] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 06:52:23 (1769082743) [ 9735.902856] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9736.043145] LustreError: 338443:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9736.069096] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9736.070821] Lustre: Skipped 2 previous similar messages [ 9737.172363] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9739.763301] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9741.070850] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9742.710437] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9743.819117] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9745.919312] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9747.786943] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9749.454803] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9757.664881] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9757.666685] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9757.669802] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9760.738770] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9762.415092] LustreError: 340785:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 9762.417752] LustreError: 340785:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 9762.452821] Lustre: server umount lustre-OST0000 complete [ 9765.344780] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9769.746924] Lustre: server umount lustre-MDT0000 complete [ 9770.944148] LustreError: 338424:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769082781 with bad export cookie 5185295073294491233 [ 9770.946297] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9770.947268] LustreError: 338424:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9771.067489] Lustre: server umount lustre-MDT0001 complete [ 9774.561525] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [ 9775.549631] Key type lgssc unregistered [ 9775.681453] LNet: 341720:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9775.683863] LNetError: 341720:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9775.694297] LNet: Removed LNI 192.168.202.152@tcp [ 9775.986129] Key type .llcrypt unregistered [ 9775.987884] Key type ._llcrypt unregistered [ 9785.639345] Key type ._llcrypt registered [ 9785.641022] Key type .llcrypt registered [ 9785.679058] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [ 9785.986886] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9786.035982] alg: No test for adler32 (adler32-zlib) [ 9786.885024] Lustre: Lustre: Build Version: 2.17.50_4_g8fdf1cc [ 9786.959904] LNet: Added LNI 192.168.202.152@tcp [8/256/0/180] [ 9788.536163] Key type lgssc registered [ 9788.871663] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9791.298395] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9791.304097] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9792.434414] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9793.584459] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9796.287583] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9796.397192] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9797.569438] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9799.290482] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9800.500743] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9802.651415] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9802.724689] LustreError: 344841:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9802.731778] LustreError: 344841:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9802.782061] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9804.569846] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9806.328717] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9811.936694] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 9811.938802] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9811.943963] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9812.450183] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9812.450390] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9812.455454] Lustre: Skipped 1 previous similar message [ 9813.853303] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.52@tcp (stopping) [ 9816.827175] LustreError: 345540:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 9816.848727] Lustre: server umount lustre-MDT0001 complete [ 9817.569871] LustreError: 343243:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9817.574859] LustreError: 343243:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9817.727355] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9818.966860] LustreError: 343240:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.52@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9820.108676] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9820.142528] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 9820.144864] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9820.157244] Lustre: 343232:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9820.253498] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9821.469843] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [ 9825.765049] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 9828.838563] LustreError: 346717:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 9828.841551] LustreError: 346717:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 9828.844287] LustreError: 343235:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 9828.859891] LustreError: 346720:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.202.152@tcp: cfg command failed: rc = -17 [ 9828.862911] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 9828.862911] [10126.205466] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.52@tcp (stopping) [10127.845601] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10127.854040] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10127.862050] Lustre: Skipped 1 previous similar message [10131.286932] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.52@tcp (stopping) [10131.297415] Lustre: Skipped 2 previous similar messages [10131.319540] LustreError: 356756:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [10131.323371] LustreError: 356756:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [10131.439710] Lustre: server umount lustre-MDT0001 complete [10132.962391] LustreError: 343244:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10132.981700] LustreError: 343244:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10133.819275] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10138.088129] LustreError: 343244:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10138.095924] LustreError: 343244:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10139.581337] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10139.650926] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10139.658199] Lustre: Found index 1 for lustre-MDT0001, updating log [10139.680912] Lustre: 343232:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10139.902314] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10142.888392] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [10145.258653] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10149.359927] LustreError: 357803:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10149.364252] LustreError: 357803:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10149.370251] LustreError: 343235:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10149.373198] LustreError: 343235:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10149.397465] LustreError: 357807:0:(obd_config.c:689:class_setup()) Device 8 already setup (type osp) [10149.401252] LustreError: 357807:0:(obd_config.c:797:class_detach()) OBD device 8 still set up [10149.404039] LustreError: 357807:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.202.152@tcp: cfg command failed: rc = -17 [10149.412516] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.152@tcp [10149.412516] [10447.335126] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10447.335312] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10447.344534] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10447.355496] Lustre: Skipped 1 previous similar message [10461.154224] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10461.212156] LustreError: 366183:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [10461.219351] LustreError: 366183:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [10461.269406] Lustre: server umount lustre-MDT0001 complete [10462.692144] LustreError: 343244:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10462.702643] LustreError: 343244:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10463.046691] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10468.037096] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10468.115383] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10468.118627] Lustre: Found index 1 for lustre-MDT0001, updating log [10468.146494] Lustre: 343233:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10468.354746] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [10471.299896] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [10473.453969] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10478.579769] LustreError: 367313:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10478.583935] LustreError: 367313:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10478.587489] LustreError: 343235:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10478.591246] LustreError: 343235:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10478.627253] LustreError: 367316:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [10478.638608] LustreError: 367316:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.202.152@tcp: cfg command failed: rc = -114 [10478.642871] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [10478.642871] [10788.835568] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10788.837293] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10788.840059] Lustre: Skipped 1 previous similar message [10788.843015] Lustre: Skipped 8 previous similar messages [10793.091256] LustreError: 375768:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [10793.094277] LustreError: 375768:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [10793.133283] Lustre: server umount lustre-MDT0000 complete [10793.953619] LustreError: 343238:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10793.973183] LustreError: 343238:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [10794.990774] LustreError: 343225:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769083805 with bad export cookie 17677669542594534000 [10794.991785] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10794.996615] LustreError: 343225:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [10799.074347] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10799.080526] Lustre: Skipped 1 previous similar message [10809.312144] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10809.355140] LustreError: 375969:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [10809.357701] LustreError: 375969:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [10809.418320] Lustre: server umount lustre-MDT0001 complete [10811.020870] Lustre: server umount lustre-OST0000 complete [10814.825356] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_hostid [10818.860974] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [10822.665803] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10825.217965] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10827.685257] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10830.719918] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10833.444464] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 07:10:43 (1769083843) [10836.244558] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10836.271115] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10836.375849] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10836.389659] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10836.429050] Lustre: lustre-MDT0000: new disk, initializing [10836.464398] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10836.476230] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10838.132857] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [10843.072209] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10843.103468] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10843.141920] Lustre: 378704:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10843.172703] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10843.175674] Lustre: Skipped 1 previous similar message [10843.215264] Lustre: lustre-MDT0001: new disk, initializing [10843.255596] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10843.269722] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10843.272968] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10844.761489] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [10848.443518] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10849.973624] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10853.276389] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10853.320133] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10853.454817] Lustre: lustre-OST0000: new disk, initializing [10853.457488] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10853.498394] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10855.865599] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_default_debug -1 all [10859.006669] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10859.011872] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10859.049384] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10859.666523] Lustre: DEBUG MARKER: oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10864.100866] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10864.102188] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10864.102675] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10864.102679] Lustre: Skipped 4 previous similar messages [10864.104958] Lustre: Skipped 2 previous similar messages [10868.803540] LustreError: 381144:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [10868.806495] LustreError: 381144:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [10868.845476] Lustre: server umount lustre-MDT0000 complete [10869.217781] LustreError: 379759:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10869.223593] LustreError: 379759:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [10870.211113] LustreError: 378696:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769083881 with bad export cookie 17677669542594535960 [10870.212829] LustreError: 379497:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [10870.213917] LustreError: MGC192.168.202.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10870.215351] LustreError: 378696:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [10870.337097] Lustre: server umount lustre-MDT0001 complete [10881.944513] Lustre: server umount lustre-OST0000 complete [10891.563257] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing set_hostid [10894.919188] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing load_modules_local [10898.310775] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10900.549357] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10902.711553] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10904.946268] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10905.535506] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 10642 sec ========= 07:11:55 (1769083915) [10906.111650] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 07:11:55 (1769083915) === [10907.310329] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 07:11:57 (1769083917) === [10921.304917] Lustre: DEBUG MARKER: oleg252-server.virtnet: executing unload_modules_local [10922.327807] Key type lgssc unregistered [10922.450324] LNet: 386646:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10922.452912] LNetError: 386646:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [10922.461330] LNet: Removed LNI 192.168.202.152@tcp [10922.789693] Key type .llcrypt unregistered [10922.791172] Key type ._llcrypt unregistered