[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-8.fc42 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 461746728 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2439 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE22D5 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 002295 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2349 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE23D9 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2411 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe22d5-0xbffe2348] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22d4] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2349-0xbffe23d8] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe23d9-0xbffe2410] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2411-0xbffe2438] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.002228] x2apic enabled [ 0.003005] Switched APIC routing to physical x2apic. [ 0.004008] kvm-guest: setup PV IPIs [ 0.007388] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008018] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009013] pid_max: default: 32768 minimum: 301 [ 0.011135] LSM: Security Framework initializing [ 0.012052] Yama: becoming mindful. [ 0.013049] SELinux: Initializing. [ 0.014065] *** VALIDATE selinux *** [ 0.022648] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027223] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028143] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029122] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031088] *** VALIDATE tmpfs *** [ 0.032325] *** VALIDATE proc *** [ 0.033247] *** VALIDATE cgroup *** [ 0.034011] *** VALIDATE cgroup2 *** [ 0.036172] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037164] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039028] Spectre V2 : User space: Vulnerable [ 0.040009] Speculative Store Bypass: Vulnerable [ 0.042645] debug: unmapping init [mem 0xffffffffb7859000-0xffffffffb7860fff] [ 0.044210] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045719] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046026] ... version: 2 [ 0.047014] ... bit width: 48 [ 0.048014] ... generic registers: 4 [ 0.049012] ... value mask: 0000ffffffffffff [ 0.050017] ... max period: 00007fffffffffff [ 0.051013] ... fixed-purpose events: 3 [ 0.052014] ... event mask: 000000070000000f [ 0.053301] rcu: Hierarchical SRCU implementation. [ 0.055397] smp: Bringing up secondary CPUs ... [ 0.056598] x86: Booting SMP configuration: [ 0.057027] .... node #0, CPUs: #1 #2 #3 [ 0.060459] smp: Brought up 1 node, 4 CPUs [ 0.062012] smpboot: Max logical packages: 1 [ 0.063019] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.217087] node 0 deferred pages initialised in 152ms [ 0.221130] devtmpfs: initialized [ 0.222296] x86/mm: Memory block size: 128MB [ 0.225155] gcov: version magic: 0x41383552 [ 0.226716] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.231163] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.234444] pinctrl core: initialized pinctrl subsystem [ 0.237240] [ 0.237844] ************************************************************* [ 0.240020] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.243021] ** ** [ 0.245016] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.248019] ** ** [ 0.251021] ** This means that this kernel is built to expose internal ** [ 0.253017] ** IOMMU data structures, which may compromise security on ** [ 0.256018] ** your system. ** [ 0.258017] ** ** [ 0.261022] ** If you see this message and you are not debugging the ** [ 0.264019] ** kernel, report this immediately to your vendor! ** [ 0.267020] ** ** [ 0.270022] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.272017] ************************************************************* [ 0.275773] NET: Registered protocol family 16 [ 0.278523] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.281078] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.285090] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.288132] cpuidle: using governor menu [ 0.289778] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.292597] PCI: Using configuration type 1 for base access [ 0.295140] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.304075] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.307048] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.310186] cryptd: max_cpu_qlen set to 1000 [ 0.313193] ACPI: Added _OSI(Module Device) [ 0.315022] ACPI: Added _OSI(Processor Device) [ 0.317013] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.319014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.321952] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.327411] ACPI: Interpreter enabled [ 0.329047] ACPI: PM: (supports S0 S3 S4 S5) [ 0.330012] ACPI: Using IOAPIC for interrupt routing [ 0.332118] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.335307] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.342662] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.344038] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.347019] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.350124] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.355605] acpiphp: Slot [2] registered [ 0.357237] acpiphp: Slot [5] registered [ 0.359216] acpiphp: Slot [6] registered [ 0.360197] acpiphp: Slot [7] registered [ 0.362179] acpiphp: Slot [8] registered [ 0.363171] acpiphp: Slot [9] registered [ 0.365192] acpiphp: Slot [10] registered [ 0.367110] acpiphp: Slot [3] registered [ 0.368123] acpiphp: Slot [4] registered [ 0.370098] acpiphp: Slot [11] registered [ 0.371103] acpiphp: Slot [12] registered [ 0.373100] acpiphp: Slot [13] registered [ 0.375134] acpiphp: Slot [14] registered [ 0.377109] acpiphp: Slot [15] registered [ 0.379115] acpiphp: Slot [16] registered [ 0.380110] acpiphp: Slot [17] registered [ 0.382120] acpiphp: Slot [18] registered [ 0.384088] acpiphp: Slot [19] registered [ 0.385108] acpiphp: Slot [20] registered [ 0.386097] acpiphp: Slot [21] registered [ 0.388081] acpiphp: Slot [22] registered [ 0.389098] acpiphp: Slot [23] registered [ 0.391095] acpiphp: Slot [24] registered [ 0.393103] acpiphp: Slot [25] registered [ 0.395136] acpiphp: Slot [26] registered [ 0.396182] acpiphp: Slot [27] registered [ 0.398128] acpiphp: Slot [28] registered [ 0.400121] acpiphp: Slot [29] registered [ 0.402159] acpiphp: Slot [30] registered [ 0.404146] acpiphp: Slot [31] registered [ 0.405085] PCI host bridge to bus 0000:00 [ 0.407027] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.410035] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.412034] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.415035] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.417031] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.420038] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.422208] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.426334] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.430259] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.440016] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.445083] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.448018] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.450014] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.452016] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.455461] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.458523] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.460043] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.462881] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.467015] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.479021] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.485018] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.491092] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.497012] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.505013] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.522025] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.530678] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.543022] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.549023] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.567024] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.578107] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.584013] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.589016] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.604022] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.616412] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.626022] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.632022] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.655022] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.670776] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.683024] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.689016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.707019] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.717496] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.728017] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.734023] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.751025] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.761446] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.765423] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.768392] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.772397] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.776624] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.786137] iommu: Default domain type: Passthrough [ 0.791879] SCSI subsystem initialized [ 0.794189] ACPI: bus type USB registered [ 0.796091] usbcore: registered new interface driver usbfs [ 0.800051] usbcore: registered new interface driver hub [ 0.803110] usbcore: registered new device driver usb [ 0.807187] pps_core: LinuxPPS API ver. 1 registered [ 0.813013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.819183] PTP clock support registered [ 0.826198] EDAC MC: Ver: 3.0.0 [ 0.828147] PCI: Using ACPI for IRQ routing [ 0.830510] NetLabel: Initializing [ 0.832013] NetLabel: domain hash size = 128 [ 0.835012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.838085] NetLabel: unlabeled traffic allowed by default [ 0.842213] vgaarb: loaded [ 0.844142] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.847014] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.862321] clocksource: Switched to clocksource kvm-clock [ 1.056182] VFS: Disk quotas dquot_6.6.0 [ 1.067298] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.072924] *** VALIDATE ramfs *** [ 1.074399] *** VALIDATE hugetlbfs *** [ 1.077604] pnp: PnP ACPI init [ 1.085225] pnp: PnP ACPI: found 6 devices [ 1.155691] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.164075] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.168293] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.172544] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.177721] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.188633] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.203025] NET: Registered protocol family 2 [ 1.206088] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.225119] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.237290] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.261688] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.270729] TCP: Hash tables configured (established 65536 bind 65536) [ 1.276918] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.284873] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.290424] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.293845] NET: Registered protocol family 1 [ 1.302428] RPC: Registered named UNIX socket transport module. [ 1.312641] RPC: Registered udp transport module. [ 1.318659] RPC: Registered tcp transport module. [ 1.320350] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.334292] NET: Registered protocol family 44 [ 1.340057] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.344756] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.346694] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.357266] pci 0000:00:01.0: quirk_isa_dma_hangs+0x0/0x20 took 10323 usecs [ 1.363826] PCI: CLS 0 bytes, default 64 [ 1.370962] Unpacking initramfs... [ 3.261468] debug: unmapping init [mem 0xffff993abcc54000-0xffff993abffbffff] [ 3.273660] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 3.279792] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 3.285400] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 4.555590] Initialise system trusted keyrings [ 4.561520] Key type blacklist registered [ 4.563435] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 4.584282] zbud: loaded [ 4.591070] *** VALIDATE nfs *** [ 4.592916] *** VALIDATE nfs4 *** [ 4.595965] pstore: using deflate compression [ 4.612728] Platform Keyring initialized [ 4.843576] NET: Registered protocol family 38 [ 4.846953] Key type asymmetric registered [ 4.853630] Asymmetric key parser 'x509' registered [ 4.858442] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 4.865522] io scheduler mq-deadline registered [ 4.868866] io scheduler kyber registered [ 4.875789] io scheduler bfq registered [ 4.883307] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 4.891649] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 4.901402] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 4.905184] ACPI: Power Button [PWRF] [ 4.914233] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 4.926256] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 4.939796] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 4.948209] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 4.967716] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 4.995840] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 5.026398] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 5.032549] Non-volatile memory driver v1.3 [ 5.034702] Linux agpgart interface v0.103 [ 5.074302] virtio_blk virtio1: [vda] 134216 512-byte logical blocks (68.7 MB/65.5 MiB) [ 5.077903] vda: detected capacity change from 0 to 68718592 [ 5.101273] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 5.105720] vdb: detected capacity change from 0 to 1073741824 [ 5.140721] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 5.166578] vdc: detected capacity change from 0 to 2621440000 [ 5.204550] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 5.215175] vdd: detected capacity change from 0 to 2621440000 [ 5.247534] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 5.251345] vde: detected capacity change from 0 to 4294967296 [ 5.269359] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 5.272442] vdf: detected capacity change from 0 to 4294967296 [ 5.280263] libphy: Fixed MDIO Bus: probed [ 5.290427] usbcore: registered new interface driver usbserial_generic [ 5.295528] usbserial: USB Serial support registered for generic [ 5.297906] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 5.305651] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 5.307661] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 5.310195] mousedev: PS/2 mouse device common for all mice [ 5.319403] rtc_cmos 00:05: RTC can wake from S4 [ 5.320248] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 5.324393] rtc_cmos 00:05: registered as rtc0 [ 5.330669] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 5.333291] intel_pstate: CPU model not supported [ 5.337672] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 5.354050] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 5.356937] hid: raw HID events driver (C) Jiri Kosina [ 5.376306] usbcore: registered new interface driver usbhid [ 5.383773] usbhid: USB HID core driver [ 5.388711] drop_monitor: Initializing network drop monitor service [ 5.400210] Initializing XFRM netlink socket [ 5.402095] NET: Registered protocol family 10 [ 5.415409] Segment Routing with IPv6 [ 5.418410] NET: Registered protocol family 17 [ 5.423452] mpls_gso: MPLS GSO support [ 5.430538] RAS: Correctable Errors collector initialized. [ 5.436238] AVX version of gcm_enc/dec engaged. [ 5.442098] AES CTR mode by8 optimization enabled [ 5.594862] sched_clock: Marking stable (5594831679, 0)->(6484210519, -889378840) [ 5.610544] registered taskstats version 1 [ 5.619939] Loading compiled-in X.509 certificates [ 5.629290] zswap: loaded using pool lzo/zbud [ 5.731327] Key type big_key registered [ 5.776249] Key type encrypted registered [ 5.780921] ima: No TPM chip found, activating TPM-bypass! [ 5.786706] ima: Allocated hash algorithm: sha1 [ 5.792437] ima: No architecture policies found [ 5.799401] evm: Initialising EVM extended attributes: [ 5.808864] evm: security.selinux [ 5.814327] evm: security.ima [ 5.815721] evm: security.capability [ 5.818757] evm: HMAC attrs: 0x1 [ 5.821770] rtc_cmos 00:05: setting system clock to 2026-01-30 06:02:53 UTC (1769752973) [ 5.831480] debug: unmapping init [mem 0xffffffffb8803000-0xffffffffb89fffff] [ 5.837340] debug: unmapping init [mem 0xffffffffb7582000-0xffffffffb7858fff] [ 5.850329] Write protecting the kernel read-only data: 28672k [ 5.860423] debug: unmapping init [mem 0xffffffffb5c03000-0xffffffffb5dfffff] [ 5.865888] debug: unmapping init [mem 0xffffffffb6514000-0xffffffffb65fffff] [ 5.968437] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 5.988944] systemd[1]: Detected virtualization kvm. [ 5.990700] systemd[1]: Detected architecture x86-64. [ 5.992350] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 6.061734] systemd[1]: No hostname configured. [ 6.066667] systemd[1]: Set hostname to . [ 6.072525] random: systemd: uninitialized urandom read (16 bytes read) [ 6.074266] systemd[1]: Initializing machine ID from random generator. [ 6.174866] random: ln: uninitialized urandom read (6 bytes read) [ 6.391981] random: systemd: uninitialized urandom read (16 bytes read) [ 6.397367] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 6.407798] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 6.421209] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. Starting Journal Service... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Setup Virtual Console... [ OK ] Reached target Slices. [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Swap. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Timers. Starting Apply Kernel Variables... [ OK ] Reached target Initrd Root Device. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 7.687809] device-mapper: uevent: version 1.0.3 [ 7.690839] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [[ 9.408954] random: fast init done  OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 9.712111] virtio_net virtio0 ens2: renamed from eth0 [ 10.192181] scsi host0: ata_piix [ 10.195208] scsi host1: ata_piix [ 10.198879] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 10.206843] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 15.346741] random: crng init done [ 15.353711] random: 7 urandom warning(s) missed due to ratelimiting [ 17.788784] dracut-initqueue[585]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 19.396826] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 21.945150] printk: systemd: 26 output lines suppressed due to ratelimiting [ 22.551843] SELinux: Disabled at runtime. [ 22.706841] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 22.723517] systemd[1]: Detected virtualization kvm. [ 22.728146] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 24.670308] systemd[1]: initrd-switch-root.service: Succeeded. [ 24.676222] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 24.685944] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 24.692261] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 24.697942] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 24.707125] systemd[1]: Starting Journal Service... Starting Journal Service... [ 24.735119] systemd[1]: Mounting Kernel Debug File System... Mounting Kernel Debug File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. Starting Remount Root and Kernel File Systems... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on udev Control Socket. [ OK ] Listening on initctl Compatibility Named Pipe. Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ 24.871713] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Mounting Huge Pages File System... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Stopped target Switch Root. Starting Apply Kernel Variables... [ OK ] Stopped target Initrd File Systems. Mounting POSIX Message Queue File System... [ OK ] Created slice system-getty.slice. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Listening on RPCbind Server Activation Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on Process Core Dump Socket. [ OK ] Stopped target Initrd Root File System. [ OK ] Reached target Paths. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 26.201294] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 27.178875] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 27.243882] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 27.557713] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 27.651786] EDAC sbridge: Ver: 1.1.2 [ 31.199969] Key type dns_resolver registered [ 31.975176] NFS: Registering the id_resolver key type [ 31.979336] Key type id_resolver registered [ 31.984239] Key type id_legacy registered [* ] A start job is running for Configur…-only root support (7s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. [ OK ] Started irqbalance daemon. Starting Login Service... Starting Network Manager... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Reached target sshd-keygen.target. Starting Restore /run/initramfs on shutdown... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg357-server login: [ 67.586485] hrtimer: interrupt took 2494376 ns [ 87.324458] libcfs: loading out-of-tree module taints kernel. [ 87.384357] Key type ._llcrypt registered [ 87.387057] Key type .llcrypt registered [ 87.492192] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_hostid [ 109.088050] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 110.323651] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 110.335593] alg: No test for adler32 (adler32-zlib) [ 111.664235] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 112.638823] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 114.479491] Key type lgssc registered [ 116.908899] Lustre: Echo OBD driver; http://www.lustre.org/ [ 137.458774] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 178.373423] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 191.669786] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 191.715870] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 193.080583] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 193.123091] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 193.269674] Lustre: lustre-MDT0000: new disk, initializing [ 193.344086] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 193.398401] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 198.857265] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 212.520930] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 212.606397] Lustre: 6499:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 212.633351] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 212.638516] Lustre: Skipped 1 previous similar message [ 212.804375] Lustre: lustre-MDT0001: new disk, initializing [ 212.890188] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 212.982899] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 213.002484] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 217.557253] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 222.915233] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 232.148860] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 232.434812] Lustre: lustre-OST0000: new disk, initializing [ 232.441834] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 232.509305] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 238.105847] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 238.119315] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 238.189127] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 238.685693] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 252.004324] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 252.101925] Lustre: lustre-OST0001: new disk, initializing [ 252.106452] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 252.158345] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 258.580712] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 258.591138] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 258.644648] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 258.740057] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 271.719968] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 283.452110] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 293.770538] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing check_logdir /tmp/testlogs/ [ 298.078138] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing yml_node [ 301.772801] Lustre: DEBUG MARKER: Client: 2.17.50.72 [ 303.950336] Lustre: DEBUG MARKER: MDS: 2.17.50.72 [ 306.195773] Lustre: DEBUG MARKER: OSS: 2.17.50.72 [ 307.478419] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri Jan 30 01:07:53 EST 2026 [ 323.547324] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 324.919934] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 346.079786] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 346.091046] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 346.097105] Lustre: Skipped 1 previous similar message [ 346.109498] Lustre: Skipped 3 previous similar messages [ 347.398202] LustreError: 12196:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 347.632542] Lustre: server umount lustre-MDT0000 complete [ 351.647784] LustreError: 6492:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769753319 with bad export cookie 8130328084281098552 [ 351.654682] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 351.663875] LustreError: 6492:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 356.320536] LustreError: 6509:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 356.321169] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 356.345038] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 356.354081] LustreError: 6509:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 356.408065] Lustre: Skipped 3 previous similar messages [ 357.854249] LustreError: 12397:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 357.861810] LustreError: 12397:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 358.222976] Lustre: server umount lustre-MDT0001 complete [ 362.147177] LustreError: 12600:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 362.152573] LustreError: 12600:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 362.403057] Lustre: server umount lustre-OST0000 complete [ 366.246436] LustreError: 12801:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 366.250926] LustreError: 12801:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 366.460851] Lustre: server umount lustre-OST0001 complete [ 374.291969] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_hostid [ 384.801574] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 419.610897] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 419.789361] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 419.812560] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 419.879770] Lustre: lustre-MDT0000: new disk, initializing [ 419.957764] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 419.975111] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 424.215527] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 434.712274] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 434.815949] Lustre: 14976:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 434.857774] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 434.860320] Lustre: Skipped 1 previous similar message [ 434.909995] Lustre: lustre-MDT0001: new disk, initializing [ 434.990130] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 435.037328] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 435.049146] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 439.008522] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 446.803220] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 451.102537] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 458.167113] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 458.373639] Lustre: lustre-OST0000: new disk, initializing [ 458.377434] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 458.466701] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 460.228740] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 460.233225] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 460.315471] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 463.340724] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 470.058369] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 474.815501] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 475.028983] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 479.478167] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 479.815866] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 485.863804] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 485.867692] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 485.880679] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 485.886622] Lustre: Skipped 1 previous similar message [ 485.915390] Lustre: Skipped 2 previous similar messages [ 488.245555] LustreError: 17906:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 488.250628] LustreError: 17906:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 488.391962] Lustre: server umount lustre-OST0000 complete [ 496.097650] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 496.110743] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 496.116257] Lustre: Skipped 1 previous similar message [ 498.385459] LustreError: 18108:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 498.396875] LustreError: 18108:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 498.571363] Lustre: server umount lustre-MDT0000 complete [ 501.217372] LustreError: 14986:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 501.243269] LustreError: 14986:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 502.443174] LustreError: 14968:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769753470 with bad export cookie 8130328084281100085 [ 502.458938] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 502.463194] LustreError: 14968:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 502.818044] Lustre: server umount lustre-MDT0001 complete [ 508.294919] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 01:11:14 (1769753474) [ 515.381603] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 515.737985] LustreError: 18835:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 515.812656] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 520.207748] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 521.184711] LustreError: 18835:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 526.306516] LustreError: 18835:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 528.596337] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 533.727766] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 538.710975] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 541.739656] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 547.759133] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 554.462286] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 559.758211] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 568.704836] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 575.457638] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 575.463046] LustreError: Skipped 1 previous similar message [ 575.467373] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 575.478359] Lustre: Skipped 1 previous similar message [ 575.493156] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 580.861816] LustreError: 21133:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 580.869744] LustreError: 21133:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 581.032593] Lustre: server umount lustre-MDT0000 complete [ 582.130466] LustreError: 18829:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 582.163075] LustreError: 18829:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 584.518534] LustreError: 18815:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769753552 with bad export cookie 8130328084281100967 [ 584.538946] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 584.544407] LustreError: 18815:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 584.884944] Lustre: server umount lustre-MDT0001 complete [ 597.927729] Lustre: server umount lustre-OST0000 complete [ 616.794886] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 626.382908] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 626.868415] LustreError: 22863:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 626.946084] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 626.960673] Lustre: Skipped 2 previous similar messages [ 632.754858] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 644.171038] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 649.320583] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 652.113554] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 657.995713] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 662.094521] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 673.102554] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 673.324874] Lustre: lustre-OST0001: new disk, initializing [ 673.337795] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 675.343863] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 675.359785] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 675.509436] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 680.123381] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 689.150083] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 692.221588] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 711.138374] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 711.153305] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 711.160034] Lustre: Skipped 4 previous similar messages [ 711.174075] Lustre: Skipped 5 previous similar messages [ 715.508687] LustreError: 26341:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 715.521313] LustreError: 26341:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 715.643750] Lustre: server umount lustre-MDT0000 complete [ 716.268155] LustreError: 25646:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 716.304546] LustreError: 25646:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 719.464047] LustreError: 22844:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769753687 with bad export cookie 8130328084281101814 [ 719.481497] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 719.487161] LustreError: 22844:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 719.913995] Lustre: server umount lustre-MDT0001 complete [ 734.338917] Lustre: server umount lustre-OST0000 complete [ 748.393284] Lustre: server umount lustre-OST0001 complete [ 766.796553] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 778.447156] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 778.717567] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 778.720770] Lustre: Skipped 3 previous similar messages [ 783.019495] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 783.850995] LustreError: 28120:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 783.868522] LustreError: 28120:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 792.853056] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 797.854543] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 801.056544] Lustre: 29234:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 808.063060] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 814.423862] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 822.989945] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 830.499364] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 838.946595] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 842.710724] Lustre: 31089:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 855.015739] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 855.015847] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 855.018614] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 855.027440] Lustre: Skipped 3 previous similar messages [ 860.410520] LustreError: 31292:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 860.419217] LustreError: 31292:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 860.633922] Lustre: server umount lustre-MDT0000 complete [ 864.280072] LustreError: 29617:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769753831 with bad export cookie 8130328084281103102 [ 864.297891] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 864.300094] LustreError: 29617:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 864.746342] Lustre: server umount lustre-MDT0001 complete [ 879.395926] Lustre: server umount lustre-OST0000 complete [ 880.610136] Lustre: 3652:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1769753832/real 1769753832] req@ffff993a063e2680 x1855720605907456/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1769753848 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 883.588064] Lustre: server umount lustre-OST0001 complete [ 893.801593] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 01:17:39 (1769753859) [ 923.545987] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 934.607961] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 935.102198] LustreError: 33875:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 935.126567] LustreError: 33875:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 940.721982] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 951.945263] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 956.328917] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 959.497308] Lustre: 34989:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 967.326692] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 975.153580] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 984.796425] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 992.456905] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 1001.417947] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1005.238404] Lustre: 36844:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1018.858163] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1018.873747] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1018.897220] Lustre: Skipped 2 previous similar messages [ 1018.904295] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1018.909116] Lustre: Skipped 7 previous similar messages [ 1022.166951] LustreError: 36996:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 1022.172607] LustreError: 36996:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 1022.334466] Lustre: server umount lustre-MDT0000 complete [ 1026.986989] LustreError: 36166:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769753994 with bad export cookie 8130328084281104215 [ 1027.000438] LustreError: 36166:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1027.007109] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1027.430100] Lustre: server umount lustre-MDT0001 complete [ 1043.236624] Lustre: server umount lustre-OST0000 complete [ 1046.495176] Lustre: 3652:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1769753998/real 1769753998] req@ffff993a031a6680 x1855720605964032/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1769754014 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1047.519528] Lustre: 3654:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1769753999/real 1769753999] req@ffff993a031a4000 x1855720605964288/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1769754015 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1047.865142] Lustre: server umount lustre-OST0001 complete [ 1059.097042] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 01:20:24 (1769754024) [ 1079.886636] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 1092.215895] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1092.698420] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1092.705856] Lustre: Skipped 7 previous similar messages [ 1098.515066] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 1108.048941] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1114.041643] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 1117.458581] Lustre: 40084:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1125.144556] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1132.921708] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 1143.020522] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1143.298884] Lustre: lustre-OST0001: Not available for connect from 0@lo (not set up) [ 1143.317375] Lustre: Skipped 4 previous similar messages [ 1151.197527] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 1160.590888] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1164.780346] Lustre: 41938:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1170.989431] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1171.005986] Lustre: Skipped 1 previous similar message [ 1175.013400] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1175.029604] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1175.051610] Lustre: Skipped 4 previous similar messages [ 1181.444671] Lustre: server umount lustre-MDT0000 complete [ 1185.807579] LustreError: 38952:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769754153 with bad export cookie 8130328084281105321 [ 1185.810648] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1185.826304] LustreError: 38952:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1192.412779] Lustre: server umount lustre-MDT0001 complete [ 1201.400655] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1210.732416] LustreError: 40449:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1210.746202] LustreError: 40449:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 28 previous similar messages [ 1215.729330] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 1222.163868] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1225.527447] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1231.854363] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1231.867307] Lustre: Skipped 1 previous similar message [ 1232.199769] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 1240.007903] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1242.537126] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1246.691872] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1248.782412] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1256.930422] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1259.451838] Lustre: server umount lustre-MDT0000 complete [ 1265.032910] LustreError: 41940:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769754232 with bad export cookie 8130328084281106476 [ 1265.035852] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1265.053557] LustreError: 41940:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1265.884550] Lustre: server umount lustre-MDT0001 complete [ 1281.043085] LustreError: 44745:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 1281.052204] LustreError: 44745:0:(obd_class.h:479:obd_check_dev()) Skipped 57 previous similar messages [ 1281.189421] Lustre: server umount lustre-OST0000 complete [ 1284.064648] Lustre: 3652:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1769754235/real 1769754235] req@ffff993b3a958e00 x1855720606060928/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1769754251 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1285.515082] Lustre: server umount lustre-OST0001 complete [ 1297.827704] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 01:24:22 (1769754262) [ 1317.993431] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 1327.916304] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1332.807931] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 1342.325260] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1347.160070] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 1350.224707] Lustre: 47435:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1356.918187] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1363.603552] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 1371.950725] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1377.789165] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 1384.844179] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1394.115360] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1409.508359] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1409.519084] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1409.526279] Lustre: Skipped 10 previous similar messages [ 1409.530419] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1409.532864] Lustre: Skipped 16 previous similar messages [ 1414.060694] Lustre: server umount lustre-MDT0000 complete [ 1417.890657] LustreError: 49346:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769754385 with bad export cookie 8130328084281107302 [ 1417.896362] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1417.906414] LustreError: 49346:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1418.399709] Lustre: server umount lustre-MDT0001 complete [ 1433.415620] Lustre: server umount lustre-OST0000 complete [ 1437.152034] Lustre: 3652:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1769754388/real 1769754388] req@ffff993b3d780a80 x1855720606121344/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1769754404 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1438.760809] Lustre: server umount lustre-OST0001 complete [ 1449.523507] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 01:26:55 (1769754415) [ 1451.944538] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1453.711106] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 01:26:59 (1769754419) [ 1464.976442] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_hostid [ 1472.918450] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 1494.702066] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1495.027971] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1495.219326] Lustre: lustre-MDT0000: new disk, initializing [ 1495.480202] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1499.692578] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 1506.723786] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1513.250171] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1513.502601] Lustre: lustre-OST0000: new disk, initializing [ 1513.507556] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1513.513255] Lustre: Skipped 1 previous similar message [ 1515.555380] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1515.567259] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1515.602473] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1519.908538] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 1526.958850] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1529.119657] Lustre: DEBUG MARKER: create test files [ 1566.691631] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1570.622202] Lustre: server umount lustre-OST0000 complete [ 1575.249326] Lustre: server umount lustre-MDT0000 complete [ 1586.846658] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 1589.695321] Key type lgssc unregistered [ 1590.073744] LNet: 54837:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1590.089457] LNetError: 54837:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1590.127036] LNet: Removed LNI 192.168.203.157@tcp [ 1591.188139] Key type .llcrypt unregistered [ 1591.192060] Key type ._llcrypt unregistered [ 1624.190858] Key type ._llcrypt registered [ 1624.193194] Key type .llcrypt registered [ 1624.309676] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 1625.511429] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1625.612596] alg: No test for adler32 (adler32-zlib) [ 1626.898436] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 1627.173357] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 1628.831162] Key type lgssc registered [ 1629.654473] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1638.072242] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1638.117049] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1639.998208] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1644.338894] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 1649.436172] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1656.293063] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1656.668442] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1662.885293] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 1664.883024] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1665.015959] Lustre: 57442:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1665.027110] Lustre: 57442:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1665.040934] Lustre: 57442:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1665.050239] Lustre: 57442:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1665.063042] Lustre: 57442:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1665.073839] Lustre: 57442:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1668.374894] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1674.203683] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1679.398939] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1690.595971] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1690.608543] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1690.638243] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1694.328723] LustreError: 58188:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1694.473770] Lustre: server umount lustre-OST0000 complete [ 1699.112959] LustreError: 58389:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1699.115911] LustreError: 58389:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1699.287663] Lustre: server umount lustre-MDT0000 complete [ 1711.350191] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 1714.257524] Key type lgssc unregistered [ 1714.662500] LNet: 58921:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1714.673794] LNetError: 58921:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1714.693300] LNet: Removed LNI 192.168.203.157@tcp [ 1715.715246] Key type .llcrypt unregistered [ 1715.718087] Key type ._llcrypt unregistered [ 1747.890791] Key type ._llcrypt registered [ 1747.894202] Key type .llcrypt registered [ 1748.015424] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 1749.364274] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1749.523029] alg: No test for adler32 (adler32-zlib) [ 1750.584834] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 1750.991681] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 1752.759356] Key type lgssc registered [ 1754.512114] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1762.063188] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1762.093448] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1763.608595] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1768.474397] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 1774.564827] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1783.004159] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1783.563954] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1788.678559] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1788.811383] Lustre: 61623:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1788.828689] Lustre: 61623:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1788.839156] Lustre: 61623:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1788.850945] Lustre: 61623:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1788.870402] Lustre: 61623:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1788.880594] Lustre: 61623:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1790.607879] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 1796.769989] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1803.504814] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1845.221285] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1845.244265] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1845.250180] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1847.893885] LustreError: 62340:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1848.036219] Lustre: server umount lustre-OST0000 complete [ 1852.517411] LustreError: 62542:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1852.524666] LustreError: 62542:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1852.854964] Lustre: server umount lustre-MDT0000 complete [ 1867.221535] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 1869.679846] Key type lgssc unregistered [ 1870.429864] LNet: 63076:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1870.438987] LNetError: 63076:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1870.455209] LNet: Removed LNI 192.168.203.157@tcp [ 1871.795975] Key type .llcrypt unregistered [ 1871.798287] Key type ._llcrypt unregistered [ 1893.377616] Key type ._llcrypt registered [ 1893.379498] Key type .llcrypt registered [ 1893.505259] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_hostid [ 1910.565646] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 1911.630985] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1911.659081] alg: No test for adler32 (adler32-zlib) [ 1913.095114] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 1913.286982] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 1914.936068] Key type lgssc registered [ 1916.463412] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1955.158266] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1955.190728] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1956.545702] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1956.582815] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1956.675494] Lustre: lustre-MDT0000: new disk, initializing [ 1956.774955] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1956.794808] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1961.365979] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 1972.815964] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1972.901269] Lustre: 66192:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1972.937865] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1972.939611] Lustre: Skipped 1 previous similar message [ 1972.988688] Lustre: lustre-MDT0001: new disk, initializing [ 1973.052849] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1973.072316] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1973.076485] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1978.096334] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 1987.273455] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1991.927961] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1999.868711] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2000.190872] Lustre: lustre-OST0000: new disk, initializing [ 2000.193988] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2000.317935] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2006.380606] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 2006.631556] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2006.637459] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2006.750961] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2013.409367] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2017.894592] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2018.139498] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2023.330513] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2023.586293] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2027.489386] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2027.505890] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2027.531623] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2027.540216] Lustre: Skipped 1 previous similar message [ 2031.683133] LustreError: 69128:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2032.068319] Lustre: server umount lustre-OST0000 complete [ 2037.738337] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2037.745064] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2037.751376] Lustre: Skipped 2 previous similar messages [ 2042.558436] LustreError: 69329:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2042.566568] LustreError: 69329:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2042.701711] Lustre: server umount lustre-MDT0000 complete [ 2046.801214] LustreError: 66186:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769755014 with bad export cookie 17550872852919957496 [ 2046.812901] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2046.821025] LustreError: 66186:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2047.091233] LustreError: 69532:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2047.113811] LustreError: 69532:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2047.458612] Lustre: server umount lustre-MDT0001 complete [ 2058.966143] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 01:37:04 (1769755024) [ 2071.099142] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 2108.269086] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_hostid [ 2117.780100] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 2160.190560] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2160.717748] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2160.782210] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2160.940701] Lustre: lustre-MDT0000: new disk, initializing [ 2161.137334] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2161.172445] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2167.624248] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 2180.829143] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2180.902677] Lustre: 73758:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2180.946627] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2180.950402] Lustre: Skipped 1 previous similar message [ 2181.015442] Lustre: lustre-MDT0001: new disk, initializing [ 2181.139556] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2181.191441] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2181.203538] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2185.941088] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 2193.966981] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2198.267930] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2205.098311] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2205.320613] Lustre: lustre-OST0000: new disk, initializing [ 2205.323879] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2205.393488] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2211.905366] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 2212.917933] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2212.927704] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2213.034219] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2220.042326] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2225.110820] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2225.374150] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2230.509951] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2230.693786] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2233.825840] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2233.829705] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2233.837390] LustreError: Skipped 2 previous similar messages [ 2233.840451] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2233.840459] Lustre: Skipped 1 previous similar message [ 2233.885850] Lustre: Skipped 1 previous similar message [ 2238.843352] LustreError: 76692:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2238.854708] LustreError: 76692:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 2238.955143] LustreError: 75906:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2238.966061] LustreError: 75906:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2239.310870] Lustre: server umount lustre-OST0000 complete [ 2244.070520] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2244.105180] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2244.106783] Lustre: Skipped 1 previous similar message [ 2244.130862] Lustre: Skipped 2 previous similar messages [ 2249.187556] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2249.199902] Lustre: Skipped 1 previous similar message [ 2249.347861] LustreError: 76893:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2249.358755] LustreError: 76893:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2249.503744] Lustre: server umount lustre-MDT0000 complete [ 2253.690060] LustreError: 73750:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769755221 with bad export cookie 17550872852919958378 [ 2253.694357] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2253.713350] LustreError: 73750:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2254.150950] Lustre: server umount lustre-MDT0001 complete [ 2264.114625] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 01:40:29 (1769755229) [ 2271.598243] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2272.027331] LustreError: 77765:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2272.038618] LustreError: 77765:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2272.112362] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2276.913202] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 2277.348483] LustreError: 77765:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2281.449543] LustreError: 77765:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2286.573385] LustreError: 77765:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2286.701701] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2291.827445] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 2298.202441] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2302.416932] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2310.056980] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2310.600887] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2310.611857] Lustre: Skipped 1 previous similar message [ 2317.487906] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 2323.584767] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2327.175417] Lustre: 80103:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting d8c179f6-fda1-4490-a0ce-67cc2bf596da at adminstrative request [ 2332.154589] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2332.183889] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2341.356230] Lustre: MGS: Client d8c179f6-fda1-4490-a0ce-67cc2bf596da (at 0@lo) reconnecting [ 2341.374130] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xf39139df08573edc to 0xf39139df0857417c [ 2341.390438] Lustre: MGC192.168.203.157@tcp: Connection restored to 0@lo (at 0@lo) [ 2363.953798] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2364.151312] Lustre: lustre-OST0001: new disk, initializing [ 2364.156028] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2370.242987] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 2373.274084] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2373.302193] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2373.429037] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2377.729619] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2383.844052] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2383.848903] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2383.867535] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2383.876663] Lustre: Skipped 1 previous similar message [ 2386.370096] LustreError: 81241:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 2386.382976] LustreError: 81241:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 2386.538668] Lustre: server umount lustre-OST0001 complete [ 2388.966146] LustreError: 79364:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2389.002762] LustreError: 79364:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2393.573106] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2393.579659] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2393.588448] LustreError: Skipped 2 previous similar messages [ 2393.641108] Lustre: Skipped 2 previous similar messages [ 2398.080646] Lustre: server umount lustre-OST0000 complete [ 2404.324957] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2404.333190] Lustre: Skipped 5 previous similar messages [ 2409.078920] LustreError: 81646:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 2409.081574] LustreError: 81646:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 2409.378614] Lustre: server umount lustre-MDT0000 complete [ 2409.441253] LustreError: 78477:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2409.448148] LustreError: 78477:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2413.695970] LustreError: 77746:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769755381 with bad export cookie 17550872852919959932 [ 2413.698723] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2413.707220] LustreError: 77746:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2414.439680] Lustre: server umount lustre-MDT0001 complete [ 2424.317408] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 01:43:10 (1769755390) [ 2426.140507] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2428.130646] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 01:43:13 (1769755393) [ 2429.699806] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2432.152443] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 01:43:17 (1769755397) [ 2433.936918] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2436.246183] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 01:43:21 (1769755401) [ 2438.256398] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2439.976680] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 01:43:25 (1769755405) [ 2441.561336] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2446.778725] loop: module loaded [ 2446.797960] loop0: detected capacity change from 0 to 4294967296 [ 2446.851446] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2448.437590] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2452.351797] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2456.451948] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2481.619244] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2481.646299] LustreError: 84495:0:(obd_config.c:859:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2481.650478] LustreError: 84495:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2481.654808] LustreError: 84495:0:(tgt_mount.c:2547:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2481.659332] LustreError: 84495:0:(super25.c:190:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2502.506342] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_hostid [ 2514.314684] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 2551.404707] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2551.702839] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2551.734418] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2551.881857] Lustre: lustre-MDT0000: new disk, initializing [ 2552.017829] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2552.027955] Lustre: Skipped 1 previous similar message [ 2552.054453] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2556.720252] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 2567.432356] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2567.576878] Lustre: 87323:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2567.822160] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2573.490329] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 2582.366257] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2587.242148] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2594.026515] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2600.525254] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 2600.542952] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2600.684235] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2607.991463] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2612.466474] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2612.688267] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2617.371653] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2617.610880] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2621.409315] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2621.413042] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2621.423102] LustreError: Skipped 1 previous similar message [ 2621.425188] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2621.437026] Lustre: Skipped 3 previous similar messages [ 2625.538134] LustreError: 90258:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2625.541398] LustreError: 90258:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2625.716549] Lustre: server umount lustre-OST0000 complete [ 2635.647446] Lustre: server umount lustre-MDT0000 complete [ 2636.778856] LustreError: 87335:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2636.804057] LustreError: 87335:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2639.095965] LustreError: 88103:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769755606 with bad export cookie 17550872852919960548 [ 2639.108769] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2639.109761] LustreError: 88103:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2639.477114] Lustre: server umount lustre-MDT0001 complete [ 2647.380916] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 01:46:53 (1769755613) [ 2653.450181] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2658.494328] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 2666.042103] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2670.346922] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 2675.972592] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2679.685763] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2686.795099] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2687.079556] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2687.087774] Lustre: Skipped 4 previous similar messages [ 2692.950834] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 2698.517580] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2702.759351] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2702.927659] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2706.885371] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2707.153314] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2713.535500] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2713.701485] Lustre: lustre-OST0001: new disk, initializing [ 2713.704743] Lustre: Skipped 2 previous similar messages [ 2713.711758] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2713.715151] Lustre: Skipped 3 previous similar messages [ 2719.307794] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 2720.413902] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2720.419947] Lustre: Skipped 2 previous similar messages [ 2720.422921] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2720.501245] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2726.030737] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2730.526509] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2730.774133] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2735.061421] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2735.228755] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2752.754533] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2774.707394] Lustre: Failing over lustre-MDT0000 [ 2775.193832] LustreError: 96237:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 2775.204276] LustreError: 96237:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2777.060407] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2777.069091] LustreError: 95678:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2777.073131] Lustre: Skipped 4 previous similar messages [ 2777.088400] LustreError: 95678:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 2777.331763] Lustre: server umount lustre-MDT0000 complete [ 2793.312516] Lustre: 64417:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1769755744/real 1769755744] req@00000000008525b3 x1855722493311616/t0(0) o400->MGC192.168.203.157@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1769755760 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2793.354185] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2800.945339] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2800.947952] LDISKFS-fs (dm-0): recovery complete [ 2800.956211] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2803.691178] LustreError: 64415:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@0000000002eb86af x1855722493320832/t0(0) o250->MGC192.168.203.157@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2804.010788] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2805.782870] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2805.787729] Lustre: 92347:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2808.162887] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 2809.333482] Lustre: 95286:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2809.360071] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2809.362987] Lustre: 95286:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 1 previous similar message [ 2809.514403] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2809.847137] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2810.177641] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2810.189169] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 1 previous similar message [ 2810.519480] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2810.524823] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 1 previous similar message [ 2810.544900] Lustre: 96991:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2810.558066] Lustre: 96991:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 7 previous similar messages [ 2811.191962] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2811.204096] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 2 previous similar messages [ 2811.527125] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2811.533181] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 2 previous similar messages [ 2812.834150] Lustre: 96991:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2812.856044] Lustre: 96991:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 6 previous similar messages [ 2813.498648] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2813.513934] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 6 previous similar messages [ 2813.847157] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2813.850915] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 6 previous similar messages [ 2817.032090] Lustre: 96991:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2817.038114] Lustre: 96991:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 12 previous similar messages [ 2817.680044] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2817.693525] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 12 previous similar messages [ 2818.016318] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2818.025972] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 12 previous similar messages [ 2825.272147] Lustre: 96991:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2825.283912] Lustre: 96991:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 25 previous similar messages [ 2825.921357] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2825.933533] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 25 previous similar messages [ 2826.263133] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2826.268743] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 25 previous similar messages [ 2841.553788] Lustre: 96991:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2841.579896] Lustre: 96991:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 50 previous similar messages [ 2842.228625] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2842.240412] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 50 previous similar messages [ 2842.569832] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2842.584414] LustreError: 96991:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 50 previous similar messages [ 2865.087126] Lustre: 96991:0:(ldlm_lib.c:2378:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2865.102293] Lustre: 96991:0:(ldlm_lib.c:2388:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2865.108481] Lustre: 96991:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 35e03292-a0f5-49fc-84f3-70c66e1f9898@192.168.203.57@tcp [ 2865.131399] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2865.144142] Lustre: 96991:0:(ldlm_lib.c:1898:abort_req_replay_queue()) @@@ aborted: req@0000000056f2f956 x1855722476342272/t0(8589934771) o101->35e03292-a0f5-49fc-84f3-70c66e1f9898@192.168.203.57@tcp:358/0 lens 592/0 e 0 to 0 dl 1769755843 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2865.177963] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2865.198084] Lustre: lustre-MDT0000: Denying connection for new client 35e03292-a0f5-49fc-84f3-70c66e1f9898 (at 192.168.203.57@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2865.209811] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2865.246247] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2865.249355] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2865.259283] Lustre: Skipped 2 previous similar messages [ 2865.304834] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2865.308528] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 2876.390178] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2876.406339] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2876.425504] Lustre: Skipped 1 previous similar message [ 2876.430643] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2876.450643] Lustre: Skipped 3 previous similar messages [ 2882.624373] Lustre: server umount lustre-OST0000 complete [ 2893.391760] Lustre: server umount lustre-OST0001 complete [ 2901.489241] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2901.507054] LustreError: Skipped 1 previous similar message [ 2904.707144] LustreError: 98059:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 2904.712935] LustreError: 98059:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 2905.102323] Lustre: server umount lustre-MDT0000 complete [ 2906.596818] LustreError: 91329:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2906.611450] LustreError: 91329:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 41 previous similar messages [ 2908.564058] LustreError: 91314:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769755876 with bad export cookie 17550872852920011760 [ 2908.577319] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2908.584904] LustreError: 91314:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2908.875733] Lustre: server umount lustre-MDT0001 complete [ 2918.341412] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 01:51:23 (1769755883) [ 2926.972706] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2926.980423] Lustre: *** cfs_fail_loc=197, val=0*** [ 2942.560880] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2947.679571] LustreError: 98953:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.157@tcp: failed processing log, type 4: rc = -110 [ 2952.870652] Lustre: *** cfs_fail_loc=197, val=0*** [ 2952.885460] Lustre: Skipped 65 previous similar messages [ 2973.474437] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2973.480668] Lustre: Skipped 2 previous similar messages [ 2979.782423] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 2985.018834] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2986.886614] Lustre: server umount lustre-OST0000 complete [ 2994.562912] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 01:52:40 (1769755960) [ 3014.663667] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3019.754257] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 3027.360307] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3031.095235] LustreError: 100490:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3031.108527] LustreError: 100490:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3031.116544] LustreError: 100490:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 16, retries 0, failed: rc = -5 [ 3031.486942] Lustre: server umount lustre-MDT0000 complete [ 3063.754286] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_hostid [ 3071.672079] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 3106.466548] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3106.839768] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3106.898773] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3107.021362] Lustre: lustre-MDT0000: new disk, initializing [ 3107.228134] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3111.901469] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 3122.493052] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3122.588964] Lustre: 104514:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3122.806477] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3127.047844] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 3134.142560] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3137.809917] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3144.156147] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3151.002468] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 3151.558721] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3158.404859] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3163.191631] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3163.533430] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3167.513636] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3167.732746] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3172.325302] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3172.328774] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3172.334164] Lustre: Skipped 5 previous similar messages [ 3172.338869] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3172.343378] LustreError: Skipped 1 previous similar message [ 3172.346704] Lustre: Skipped 8 previous similar messages [ 3175.369862] LustreError: 107448:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3175.381774] LustreError: 107448:0:(obd_class.h:479:obd_check_dev()) Skipped 33 previous similar messages [ 3175.641216] Lustre: server umount lustre-OST0000 complete [ 3185.659791] Lustre: server umount lustre-MDT0000 complete [ 3187.698557] LustreError: 104520:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3187.724189] LustreError: 104520:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 3189.582490] LustreError: 105302:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769756157 with bad export cookie 17550872852920022351 [ 3189.594157] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3189.599922] LustreError: 105302:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3189.971533] Lustre: server umount lustre-MDT0001 complete [ 3197.240693] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 01:56:03 (1769756163) [ 3205.618834] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 3208.001714] Key type lgssc unregistered [ 3208.290431] LNet: 108579:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3208.300276] LNetError: 108579:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3208.338734] LNet: Removed LNI 192.168.203.157@tcp [ 3209.230238] Key type .llcrypt unregistered [ 3209.242020] Key type ._llcrypt unregistered [ 3224.315765] Key type ._llcrypt registered [ 3224.321299] Key type .llcrypt registered [ 3224.418655] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_hostid [ 3237.097480] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 3237.931419] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3237.963955] alg: No test for adler32 (adler32-zlib) [ 3239.004858] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 3239.207667] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 3240.887430] Key type lgssc registered [ 3241.978751] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3278.606616] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3278.625443] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3279.898625] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3279.930697] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3280.001463] Lustre: lustre-MDT0000: new disk, initializing [ 3280.069418] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3280.094927] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3284.646389] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 3295.108891] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3295.642144] Lustre: lustre-OST0000: new disk, initializing [ 3295.644908] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3295.650869] Lustre: Skipped 1 previous similar message [ 3295.749627] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3302.516150] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 3303.007128] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3303.025452] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3303.057602] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3311.740226] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3311.951610] Lustre: lustre-OST0001: new disk, initializing [ 3311.957021] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3312.057978] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3312.404729] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3312.412822] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3312.452966] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3316.977776] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 3327.970493] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3327.980642] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3332.024446] LustreError: 114115:0:(obd_class.h:479:obd_check_dev()) Device 17 not setup [ 3332.289271] Lustre: server umount lustre-MDT0000 complete [ 3337.906346] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3342.555500] LustreError: 111881:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769756310 with bad export cookie 13659142785364122774 [ 3342.567047] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3342.620346] LustreError: 114712:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 3342.623301] LustreError: 114712:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3342.662335] Lustre: server umount lustre-OST0000 complete [ 3345.952325] LustreError: 114912:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 3345.957387] LustreError: 114912:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 3346.072205] Lustre: server umount lustre-OST0001 complete [ 3354.605176] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 01:58:40 (1769756320) [ 3364.842359] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 3385.918706] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 01:59:11 (1769756351) [ 3427.186345] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_hostid [ 3438.273416] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 3473.812757] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3474.109287] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3474.163510] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3474.255719] Lustre: lustre-MDT0000: new disk, initializing [ 3474.350426] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3474.369550] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3478.986164] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 3489.804958] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3489.981317] Lustre: 120210:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3490.079953] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3490.086305] Lustre: Skipped 1 previous similar message [ 3490.152310] Lustre: lustre-MDT0001: new disk, initializing [ 3490.269176] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3490.347582] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3490.365956] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3494.888670] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 3502.096289] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3505.223609] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3511.624965] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3511.943656] Lustre: lustre-OST0000: new disk, initializing [ 3511.952975] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3512.121722] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3517.774991] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 3518.597833] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3518.616134] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3518.766856] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3527.232102] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3532.928206] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3533.246610] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3538.219780] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3538.493846] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3544.553212] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3544.562686] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3544.569177] LustreError: Skipped 1 previous similar message [ 3544.572079] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3544.572088] Lustre: Skipped 1 previous similar message [ 3544.612712] Lustre: Skipped 2 previous similar messages [ 3546.571117] LustreError: 123153:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3546.575979] LustreError: 123153:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3546.804142] Lustre: server umount lustre-OST0000 complete [ 3551.713488] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3551.723097] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3551.741762] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3551.743414] Lustre: Skipped 1 previous similar message [ 3554.796739] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3554.812421] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3556.185685] LustreError: 123355:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 3556.198201] LustreError: 123355:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3556.461734] Lustre: server umount lustre-MDT0000 complete [ 3561.413485] LustreError: 120200:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769756529 with bad export cookie 13659142785364124006 [ 3561.415963] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3561.424306] LustreError: 120200:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3561.859581] Lustre: server umount lustre-MDT0001 complete [ 3572.431926] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 02:02:18 (1769756538) [ 3579.085978] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3579.832684] LustreError: 124227:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3579.846660] LustreError: 124227:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3579.904941] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3584.169646] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 3584.994306] LustreError: 124227:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3590.112317] LustreError: 124227:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3592.288227] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3597.275546] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 3603.820982] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3608.164449] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3615.897953] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3616.452584] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3616.463194] Lustre: Skipped 1 previous similar message [ 3623.555563] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 3629.688759] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3633.782497] Lustre: *** cfs_fail_loc=159, val=0*** [ 3650.110303] Lustre: lustre-MDT0000: Client c2dffefb-1381-47f9-a8d9-c328653e8a4b (at 192.168.203.57@tcp) reconnecting [ 3652.634965] Lustre: *** cfs_fail_loc=159, val=0*** [ 3652.636744] Lustre: Skipped 5 previous similar messages [ 3669.116680] Lustre: lustre-MDT0000: Client c2dffefb-1381-47f9-a8d9-c328653e8a4b (at 192.168.203.57@tcp) reconnecting [ 3676.141528] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3676.146312] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3676.159278] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3676.161894] Lustre: Skipped 1 previous similar message [ 3676.170143] Lustre: Skipped 2 previous similar messages [ 3681.093098] LustreError: 126707:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3681.097375] LustreError: 126707:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 3681.258333] LustreError: 125823:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3681.280397] LustreError: 125823:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3681.416552] Lustre: server umount lustre-OST0000 complete [ 3686.369509] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3686.385546] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3686.391693] Lustre: Skipped 1 previous similar message [ 3686.407972] Lustre: Skipped 1 previous similar message [ 3691.781404] Lustre: server umount lustre-MDT0000 complete [ 3695.145598] LustreError: 126488:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769756662 with bad export cookie 13659142785364124888 [ 3695.150247] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3695.155481] LustreError: 126488:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3695.376265] Lustre: server umount lustre-MDT0001 complete [ 3706.165464] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 3709.600510] Key type lgssc unregistered [ 3709.988669] LNet: 127644:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3710.004541] LNetError: 127644:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3710.043385] LNet: Removed LNI 192.168.203.157@tcp [ 3711.251472] Key type .llcrypt unregistered [ 3711.254422] Key type ._llcrypt unregistered [ 3736.259471] Key type ._llcrypt registered [ 3736.262489] Key type .llcrypt registered [ 3736.339874] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 3737.322092] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3737.434528] alg: No test for adler32 (adler32-zlib) [ 3738.736297] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 3739.134530] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 3740.903125] Key type lgssc registered [ 3742.302966] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3750.593805] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3750.614334] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3752.261948] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3757.186031] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 3761.635231] LustreError: 129042:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3761.680556] LustreError: 129042:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3766.159876] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3766.433533] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3771.309383] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 3776.778376] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3780.499931] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3787.377472] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3787.856541] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3794.215944] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 3797.107537] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3799.480583] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3808.052196] Lustre: *** cfs_fail_loc=159, val=0*** [ 3823.692065] Lustre: lustre-MDT0000: Client 2fbeef0c-0871-4526-9236-424ba7a239d3 (at 192.168.203.57@tcp) reconnecting [ 3826.022641] Lustre: *** cfs_fail_loc=159, val=0*** [ 3841.612677] Lustre: lustre-MDT0001: Client 2fbeef0c-0871-4526-9236-424ba7a239d3 (at 192.168.203.57@tcp) reconnecting [ 3843.513924] Lustre: *** cfs_fail_loc=159, val=0*** [ 3843.533775] Lustre: Skipped 3 previous similar messages [ 3859.013832] Lustre: lustre-MDT0001: Client 2fbeef0c-0871-4526-9236-424ba7a239d3 (at 192.168.203.57@tcp) reconnecting [ 3863.746587] Lustre: 131747:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3865.904494] Lustre: *** cfs_fail_loc=159, val=0*** [ 3865.914222] Lustre: Skipped 4 previous similar messages [ 3882.609163] Lustre: lustre-MDT0000: Client dddb2055-9dd5-490f-a54e-e38210807020 (at 192.168.203.57@tcp) reconnecting [ 3884.602080] Lustre: *** cfs_fail_loc=159, val=0*** [ 3884.603529] Lustre: Skipped 14 previous similar messages [ 3901.047748] Lustre: lustre-MDT0000: Client dddb2055-9dd5-490f-a54e-e38210807020 (at 192.168.203.57@tcp) reconnecting [ 3903.290646] Lustre: 132036:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3920.358100] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3920.365983] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3920.397169] LustreError: Skipped 1 previous similar message [ 3920.402096] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3920.415057] Lustre: Skipped 1 previous similar message [ 3924.681871] LustreError: 132161:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3924.788174] Lustre: server umount lustre-OST0000 complete [ 3930.591895] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3930.594784] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3930.603429] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3930.629488] Lustre: Skipped 2 previous similar messages [ 3930.656303] Lustre: Skipped 1 previous similar message [ 3934.392974] LustreError: 132362:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3934.407361] LustreError: 132362:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3934.673708] Lustre: server umount lustre-MDT0000 complete [ 3935.712769] LustreError: 129039:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3935.745232] LustreError: 129039:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3938.484913] LustreError: 129022:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769756906 with bad export cookie 13575479058688333552 [ 3938.491334] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3938.495060] LustreError: 129022:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3938.614799] LustreError: 132564:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3938.628043] LustreError: 132564:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3938.817861] Lustre: server umount lustre-MDT0001 complete [ 3948.949805] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 3951.698703] Key type lgssc unregistered [ 3952.009784] LNet: 133096:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3952.020502] LNetError: 133096:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3953.073903] LNet: Removed LNI 192.168.203.157@tcp [ 3954.472775] Key type .llcrypt unregistered [ 3954.475139] Key type ._llcrypt unregistered [ 3981.140542] Key type ._llcrypt registered [ 3981.145843] Key type .llcrypt registered [ 3981.292078] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 3983.009974] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3983.175096] alg: No test for adler32 (adler32-zlib) [ 3984.387389] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 3984.673198] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 3986.431323] Key type lgssc registered [ 3987.431248] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3994.426324] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3994.437934] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3995.777918] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3999.265677] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 4005.350422] LustreError: 134491:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4005.389554] LustreError: 134491:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4008.133233] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4008.501460] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4013.383599] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 4019.652569] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4023.868531] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4030.371252] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4030.747568] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4033.846123] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4036.273580] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 4041.012634] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4042.473830] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4049.373294] Lustre: 136731:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4053.131981] Lustre: 136780:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4054.864605] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4055.772529] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4059.472120] Lustre: 136927:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4060.374356] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4061.147363] Lustre: 137027:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4066.784208] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4066.793953] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4066.813499] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4069.859151] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4069.868363] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4069.872910] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4069.890396] Lustre: Skipped 1 previous similar message [ 4070.975131] LustreError: 137134:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4071.106083] Lustre: server umount lustre-OST0000 complete [ 4074.979448] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4074.988249] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4074.999462] Lustre: Skipped 1 previous similar message [ 4075.018600] Lustre: Skipped 1 previous similar message [ 4080.098138] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4080.105090] Lustre: Skipped 1 previous similar message [ 4080.891813] LustreError: 137336:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4080.907938] LustreError: 137336:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4083.113872] LustreError: 137336:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4083.126205] LustreError: 137336:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4083.188464] Lustre: server umount lustre-MDT0000 complete [ 4085.217386] LustreError: 134491:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4086.556041] LustreError: 134473:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769757054 with bad export cookie 10361630257040866724 [ 4086.560816] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4086.575439] LustreError: 134473:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4086.711469] LustreError: 137537:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4086.716922] LustreError: 137537:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 4086.874366] Lustre: server umount lustre-MDT0001 complete [ 4096.110278] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 4100.630247] Key type lgssc unregistered [ 4100.995601] LNet: 138069:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4101.013605] LNetError: 138069:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4101.040329] LNet: Removed LNI 192.168.203.157@tcp [ 4101.911686] Key type .llcrypt unregistered [ 4101.914458] Key type ._llcrypt unregistered [ 4125.191938] Key type ._llcrypt registered [ 4125.193737] Key type .llcrypt registered [ 4125.270763] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 4126.131707] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4126.180854] alg: No test for adler32 (adler32-zlib) [ 4127.291141] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 4127.572381] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 4129.295281] Key type lgssc registered [ 4130.451599] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4137.494929] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4137.517354] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4139.015029] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4143.303478] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 4149.217399] LustreError: 139445:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4149.226041] LustreError: 139445:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4151.515322] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4151.941923] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4156.804710] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 4163.183440] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4167.007749] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4174.129371] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4174.576387] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4176.628122] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4179.803171] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 4183.791529] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4184.600308] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4190.975388] Lustre: *** cfs_fail_loc=159, val=0*** [ 4206.640746] Lustre: lustre-MDT0000: Client ca45bdc2-b2f5-40d7-acc3-767e39043be5 (at 192.168.203.57@tcp) reconnecting [ 4212.207992] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4212.208564] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4212.219530] LustreError: Skipped 2 previous similar messages [ 4212.235669] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4213.259115] LustreError: 141816:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4213.489659] Lustre: server umount lustre-OST0000 complete [ 4218.336461] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4218.346756] LustreError: Skipped 5 previous similar messages [ 4218.357447] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4218.370448] Lustre: Skipped 1 previous similar message [ 4218.382516] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4218.386214] Lustre: Skipped 1 previous similar message [ 4219.872043] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4219.873026] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4223.154689] LustreError: 142027:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4223.162971] LustreError: 142027:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4223.402885] Lustre: server umount lustre-MDT0000 complete [ 4224.997690] LustreError: 140158:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4225.010875] LustreError: 140158:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4227.110596] LustreError: 139425:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769757194 with bad export cookie 10370000734703362652 [ 4227.120590] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4227.123728] LustreError: 139425:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4227.305107] LustreError: 142229:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4227.311270] LustreError: 142229:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4227.536139] Lustre: server umount lustre-MDT0001 complete [ 4236.838177] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 4239.404838] Key type lgssc unregistered [ 4239.713336] LNet: 142761:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4239.719104] LNetError: 142761:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4239.748465] LNet: Removed LNI 192.168.203.157@tcp [ 4240.635815] Key type .llcrypt unregistered [ 4240.638048] Key type ._llcrypt unregistered [ 4263.867619] Key type ._llcrypt registered [ 4263.873542] Key type .llcrypt registered [ 4263.969334] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 4264.847384] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4264.960727] alg: No test for adler32 (adler32-zlib) [ 4266.040921] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 4266.301877] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 4267.967211] Key type lgssc registered [ 4269.186755] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4276.647233] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4276.660680] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4278.135033] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4282.242682] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 4287.485252] LustreError: 144159:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4287.510028] LustreError: 144159:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4291.025895] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4291.465607] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4295.549626] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 4301.196717] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4304.622414] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4310.872562] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4311.246460] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4316.398229] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4317.510616] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 4322.566315] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4323.435783] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4343.513354] Lustre: lustre-OST0000: evicting afec0437-48b1-4fdb-ba5d-e3dc78f0eae0 (at 192.168.203.57@tcp) by administrative request [ 4343.521736] Lustre: Skipped 1 previous similar message [ 4372.962442] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4372.976561] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4373.000358] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4373.985817] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4373.987458] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4374.008348] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4374.031288] Lustre: Skipped 1 previous similar message [ 4377.440655] LustreError: 146938:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4377.622429] Lustre: server umount lustre-OST0000 complete [ 4383.718211] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4383.729172] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4383.748306] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4387.105475] LustreError: 147141:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4387.119320] LustreError: 147141:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4387.257597] Lustre: server umount lustre-MDT0000 complete [ 4389.346974] LustreError: 144154:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4389.364094] LustreError: 144154:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4390.489144] LustreError: 144139:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769757358 with bad export cookie 7171362871011761234 [ 4390.497189] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4390.509715] LustreError: 144139:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4390.640212] LustreError: 147342:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4390.650867] LustreError: 147342:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4390.843255] Lustre: server umount lustre-MDT0001 complete [ 4400.014368] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 4402.438704] Key type lgssc unregistered [ 4402.696918] LNet: 147874:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4402.706333] LNetError: 147874:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4402.732387] LNet: Removed LNI 192.168.203.157@tcp [ 4403.416631] Key type .llcrypt unregistered [ 4403.418649] Key type ._llcrypt unregistered [ 4475.994370] Key type ._llcrypt registered [ 4475.996699] Key type .llcrypt registered [ 4476.092138] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 4477.161365] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4477.227319] alg: No test for adler32 (adler32-zlib) [ 4478.290338] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 4478.544612] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 4480.231187] Key type lgssc registered [ 4481.334223] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4488.780512] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4488.807769] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4490.361620] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4494.660879] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 4499.941177] LustreError: 150358:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4499.963972] LustreError: 150358:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4503.007460] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4503.417668] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4507.882134] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 4514.009733] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4518.028802] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4524.858943] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4525.211903] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4528.336404] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4531.842406] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 4534.451616] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4537.750218] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4549.088437] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4549.091368] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4549.107151] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4550.118912] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4550.121634] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4550.127966] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4553.807107] LustreError: 152608:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4553.973726] Lustre: server umount lustre-OST0000 complete [ 4559.841046] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4559.854105] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4559.862838] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4559.869528] Lustre: Skipped 1 previous similar message [ 4563.897337] LustreError: 152809:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4563.911499] LustreError: 152809:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4564.072807] Lustre: server umount lustre-MDT0000 complete [ 4565.473675] LustreError: 150353:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4565.503874] LustreError: 150353:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4567.254985] LustreError: 150341:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769757534 with bad export cookie 12406712923021179392 [ 4567.270403] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4567.430963] LustreError: 153011:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4567.438782] LustreError: 153011:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4567.627393] Lustre: server umount lustre-MDT0001 complete [ 4578.507202] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 4581.101888] Key type lgssc unregistered [ 4581.328914] LNet: 153545:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4581.338694] LNetError: 153545:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4581.360691] LNet: Removed LNI 192.168.203.157@tcp [ 4582.200267] Key type .llcrypt unregistered [ 4582.203379] Key type ._llcrypt unregistered [ 4604.831974] Key type ._llcrypt registered [ 4604.833868] Key type .llcrypt registered [ 4604.920803] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 4605.808937] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4605.834269] alg: No test for adler32 (adler32-zlib) [ 4607.017577] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 4607.171801] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 4608.871409] Key type lgssc registered [ 4610.242147] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4624.260475] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 02:19:50 (1769757590) [ 4639.853032] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_hostid [ 4648.453417] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 4681.317485] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4681.335413] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4682.571676] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4682.594973] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4682.651145] Lustre: lustre-MDT0000: new disk, initializing [ 4682.712374] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4682.724352] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4686.337564] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 4695.522027] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4695.676194] Lustre: 157754:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4695.719889] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4695.723160] Lustre: Skipped 1 previous similar message [ 4695.835081] Lustre: lustre-MDT0001: new disk, initializing [ 4696.021048] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4696.074712] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4696.083311] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4699.818407] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 4707.193413] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4710.596899] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4716.779136] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4717.012147] Lustre: lustre-OST0000: new disk, initializing [ 4717.018595] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4717.105742] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4722.736212] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 4726.327417] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4726.348203] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4726.421924] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4729.509707] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4742.120500] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4742.136727] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4742.142356] LustreError: Skipped 1 previous similar message [ 4742.143129] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4742.151065] Lustre: Skipped 1 previous similar message [ 4742.181390] Lustre: Skipped 1 previous similar message [ 4747.235844] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4747.241341] Lustre: Skipped 1 previous similar message [ 4752.351093] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4752.357150] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4752.375244] Lustre: Skipped 1 previous similar message [ 4752.422625] LustreError: 160333:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4752.547650] Lustre: server umount lustre-OST0000 complete [ 4757.472093] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4757.477699] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4757.480883] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4757.498686] Lustre: Skipped 1 previous similar message [ 4757.511373] Lustre: Skipped 1 previous similar message [ 4762.291332] LustreError: 160537:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 4762.301920] LustreError: 160537:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4762.572640] Lustre: server umount lustre-MDT0000 complete [ 4765.851741] LustreError: 157745:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769757733 with bad export cookie 16243990670716688921 [ 4765.851969] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4765.857398] LustreError: 157745:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4765.969440] LustreError: 160739:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4765.974158] LustreError: 160739:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4766.118760] Lustre: server umount lustre-MDT0001 complete [ 4775.273611] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 4777.829496] Key type lgssc unregistered [ 4778.128631] LNet: 161271:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4778.134383] LNetError: 161271:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4778.149307] LNet: Removed LNI 192.168.203.157@tcp [ 4778.917366] Key type .llcrypt unregistered [ 4778.920459] Key type ._llcrypt unregistered [ 4798.631612] Key type ._llcrypt registered [ 4798.635271] Key type .llcrypt registered [ 4798.745930] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 4799.463827] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4799.521574] alg: No test for adler32 (adler32-zlib) [ 4800.498779] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 4800.738829] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 4802.367444] Key type lgssc registered [ 4803.351833] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4811.156780] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4811.196915] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4812.652886] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4816.784078] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 4823.008828] LustreError: 162644:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4823.017730] LustreError: 162644:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4823.726095] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4824.049671] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4827.163734] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 4831.249594] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4834.002723] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4840.110467] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4840.507223] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4846.574586] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 4851.648178] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4857.906272] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4866.751019] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4866.770492] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4866.783284] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4866.789430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4866.797542] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4867.561210] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4867.573970] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4867.581820] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4867.586958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.594742] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4868.268457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.886657] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4868.890426] Lustre: Skipped 1 previous similar message [ 4868.902954] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4868.910933] Lustre: Skipped 1 previous similar message [ 4868.914378] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4868.918600] Lustre: Skipped 1 previous similar message [ 4868.923405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.928893] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4868.934035] Lustre: Skipped 1 previous similar message [ 4869.597161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4870.398284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.248875] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4871.254811] Lustre: Skipped 2 previous similar messages [ 4871.269615] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4871.277751] Lustre: Skipped 2 previous similar messages [ 4871.280689] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4871.283702] Lustre: Skipped 2 previous similar messages [ 4871.285928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.294131] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4871.298071] Lustre: Skipped 2 previous similar messages [ 4871.935625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4872.679061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.439963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4874.095124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4874.974227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.643355] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4875.647513] Lustre: Skipped 5 previous similar messages [ 4875.660487] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4875.667480] Lustre: Skipped 5 previous similar messages [ 4875.670752] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4875.674631] Lustre: Skipped 5 previous similar messages [ 4875.677888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.685155] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4875.689346] Lustre: Skipped 5 previous similar messages [ 4876.408528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4877.131516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4877.958801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4878.803300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4879.450094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.279990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.103281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.995367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.818827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.520593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.102456] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4884.107629] Lustre: Skipped 10 previous similar messages [ 4884.126479] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4884.135321] Lustre: Skipped 10 previous similar messages [ 4884.142428] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4884.149044] Lustre: Skipped 10 previous similar messages [ 4884.159764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.187801] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4884.192931] Lustre: Skipped 10 previous similar messages [ 4884.915472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.537933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4886.336413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4886.970673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.661993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.356761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.164670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.979880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.778092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.461091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.411673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.201420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.159419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.936928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.758465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.627239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.475872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.183843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.016931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.752671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.494817] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4900.503538] Lustre: Skipped 20 previous similar messages [ 4900.520270] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4900.531622] Lustre: Skipped 20 previous similar messages [ 4900.535640] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4900.540217] Lustre: Skipped 20 previous similar messages [ 4900.543958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.551225] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4900.557364] Lustre: Skipped 20 previous similar messages [ 4901.371138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.164532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.967654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.731276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.420153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.289085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.031503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.777575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.466848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.187541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.909656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.662647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.403741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.172616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.920504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.740633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4913.490372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.321153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.036748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.696184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.329515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.159569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.879025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.700101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.433429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.195360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.962447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4921.689853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4922.353066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.076308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.624736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.338414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.986142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.653680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4926.404113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.067882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.676693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.378154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.070269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.745792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.519402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.307480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.018234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.705256] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4932.711352] Lustre: Skipped 43 previous similar messages [ 4932.726765] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4932.732233] Lustre: Skipped 43 previous similar messages [ 4932.736335] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4932.739767] Lustre: Skipped 43 previous similar messages [ 4932.743233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.755872] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4932.760539] Lustre: Skipped 43 previous similar messages [ 4933.466565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.290532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.097983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.908124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.682863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.468188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.252995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.000596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.766310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.573475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.262161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.086649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.941165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.863147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.639165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.380501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.192094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.046292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.996506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.862717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.654462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.351169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.099480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.695863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.332573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.144997] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.911314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.660270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.397802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.277408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.987844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.835205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.581643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.353978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.361611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.271886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.128178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.917978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.678391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.580060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.392866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.252493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.980963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.674756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.378042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.136050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.878220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.606045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.360380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.182251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.973241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.752885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.594063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.408508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.306967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.194660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.075110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.929325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.692949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.503746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.395680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.275785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.903114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.647967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.451426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.374040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.227557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.972330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.698057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.347794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.070798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.777627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.449540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.192747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.923891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.686388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.520630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.281793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.959912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.704275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.433829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.140672] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4997.143900] Lustre: Skipped 81 previous similar messages [ 4997.154743] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4997.163539] Lustre: Skipped 81 previous similar messages [ 4997.166123] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4997.171252] Lustre: Skipped 81 previous similar messages [ 4997.175292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.182817] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4997.189708] Lustre: Skipped 81 previous similar messages [ 4997.889884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.691318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.656178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.350851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.041864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.883915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.532740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.305847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.079389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.774780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.330818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.910496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.573965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.099262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.764183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.380857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.981884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.722102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.388492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.147165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.790885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.392132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.089351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.726960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.380831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.005392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.652763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.276829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.001472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.674077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.436913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.218740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.940567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.712520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.424215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.002959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.671534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.346684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.956375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.537939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.058058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.616064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.094201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.742108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.272509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.821382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.341454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.900907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.639474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.392110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.058346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.709396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.291803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.934036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.494490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.225171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.966102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.651883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.232816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.942446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.544602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.167546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.813683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.430069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.152124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.869784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.495168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.161739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.871658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.533283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.052942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.656145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.339795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.964153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.632261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.324650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.018159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.659248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.343782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.087286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.735537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.464663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.127763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.866926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.473630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.169245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.791268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.560896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.101462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.716235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.353579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.111594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.714100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.335181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.859704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.483953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.059706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.668728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.190361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.814644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.503129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.209886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.875644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.695734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.415400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.162030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.892912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.636898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.293610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.115254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.818036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.560146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.284116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.001321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.711482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.485569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.185582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.899654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.623257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.334471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.017985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.643591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.356131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.060543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.826851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.546400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.409837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.108446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.908532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.665100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.503227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.229725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.930308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.583962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.214307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.945166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.707798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.276417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.149846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.854234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.581792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.288629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.007914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.643355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.359163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.013776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.710439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.300600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.909859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.566069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.284360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.955983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.643094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.313192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.988064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.533832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.181785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.864801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.350432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.971709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.752301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.406076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.010474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.839274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.540461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.271762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.943407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.562791] LustreError: 162664:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5110.568133] LustreError: 162664:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5110.577923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.136686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.933471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.591989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.371856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.110524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.860644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.541842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.275334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.964187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.698921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.383172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.074925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.724205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.370709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.961311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.628652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.359484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.974867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.766486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.464982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.128230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.802982] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5125.816261] Lustre: Skipped 189 previous similar messages [ 5125.830372] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5125.840613] Lustre: Skipped 189 previous similar messages [ 5125.844697] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5125.850433] Lustre: Skipped 189 previous similar messages [ 5125.852458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.871182] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5125.875327] Lustre: Skipped 189 previous similar messages [ 5126.548639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.283078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.026531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.757211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.574379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.428155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.101163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.748203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.417926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.128813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.683949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.234423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.743160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.227941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.789965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.317448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.830587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.486548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.993788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.595596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.169095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.733641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.220896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.854584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.342559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.889941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.504729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.973372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.567919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.164901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.762717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.357415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.950416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.463136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.027702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.673049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.338571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.926476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.587581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.164866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.704620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.261433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.851743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.559526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.181205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.886577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.543095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.059030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.638038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.216850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.745805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.318436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.906611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.694055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.260265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.792314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.337707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.914588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.537039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.329734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.914589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.542050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.154771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.850649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.494595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.054935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.668768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.286279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.899859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.544632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.047788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.635589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.306564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.052530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.560861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.135706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.643421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.138332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.600342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.125649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.568510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.043949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.579196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.151895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.765433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.296846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.831849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.453868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.116602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.699404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.420218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.059821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.732436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.309600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.884267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.424038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.967338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.491421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.094423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.671728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.180448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.681916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.178179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.752456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.352744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.967513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.679167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.361024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.990252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.580431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.112820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.656648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.257772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.839957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.438154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.091395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.826356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.436311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.114940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.772196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.438957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.085129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.734132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.305706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.949413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.498638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.049393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.631954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.151784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.660963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.183251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.744155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.264694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.817202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.504035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.097445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.712190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.378481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.979892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.603500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.225566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.920627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.571480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.113847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.697213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.319620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.886961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.514284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.234783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.898200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.561166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.221615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.923452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.579584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.264303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.831394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.509387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.192351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.877556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.488655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.099280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.727580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.442351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.147723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.747901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.431887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.173912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.803479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.520254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.115597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.727281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.355102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.131844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.761168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.371071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.960623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.675944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.344230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.034560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.666528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.313544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.928334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.586461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.105562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.682724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.313445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.925464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.596665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.181677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.779094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.381832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.976124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.616074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.210460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.856501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.557669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.148500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.809489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.468471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.123397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.791513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.470093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.057316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.598109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.248198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.925337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.646044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.336870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.057863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.775945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.353093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.006532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.726208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.483476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.074586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.706624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.344171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.042692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.661057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.308562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.926046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.540755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.142724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.697227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.324948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.975111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.691186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.246690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.880283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.502134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.086875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.737575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.330690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.022512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.626166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.263577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.838646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.585623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.287104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.982081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.683408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.497765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.135232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.769983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.352239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.912480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.419579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.992266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.549807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.140892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.711499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.370700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.977523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.583996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.104723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.824626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.589740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.369760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.896609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.510643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.074523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.773584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.443467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.230959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.984820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.601155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.159628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.739984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.366885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.015967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.530067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.202439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.803904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.375721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.881210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.402185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.995383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.520570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.105773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.652554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.191617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.735053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.309593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.912578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.560680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.163410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.711576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.232528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.726256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.187840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.730754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.275828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.846481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.360272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.944765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.541309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.234345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.934946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.593372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.259187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.946841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.532664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.167929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.706218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.334943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.956956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.539668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.104833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.691441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.286512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.829704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.408078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.941950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.585418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.221624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.777551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.358562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.963779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.634963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.212891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.822354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.367202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.962839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.436302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.924593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.511446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.038260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.571728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.120545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.676581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.261474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.858030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.450728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.035797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.613882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.232810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.641541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.204912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.714743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.232884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.745895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.275772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.914537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.511664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.122454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.696663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.260928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.768434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.280916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.870583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.512937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.107217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.562115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.146283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.592359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.109672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.689150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.166178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.609331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.036752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.459892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.065918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.634219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.419812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.373280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.464638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.217030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.019944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.676451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.377715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.971963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.637926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.177404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.688849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.360205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.242710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.152363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.127680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.827589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.553771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.244452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.997504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.597719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.361605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.037033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.702335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.428595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.161204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.788430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.455683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.040760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.507060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.952635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.456257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.948030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.336139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.709440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.085759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.485492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.858428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.243892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.614972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.997166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.378633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.747680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.226717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.726038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.103885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.488188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.872076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.255749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.727844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.144312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.673748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.157659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.602832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.019745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.430045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.817119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.242362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.640357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.051952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.420388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.842075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.289532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.688076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.099083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.646853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.168868] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5382.172680] Lustre: Skipped 428 previous similar messages [ 5382.186020] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5382.191037] Lustre: Skipped 428 previous similar messages [ 5382.193234] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5382.197847] Lustre: Skipped 428 previous similar messages [ 5382.200587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.205307] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5382.210189] Lustre: Skipped 428 previous similar messages [ 5382.834162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.488426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.041183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.670881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.373355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.108331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.762038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.325048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.950689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.507573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.063079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.620577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.220903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.731292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.265654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.738606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.276890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.737393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.258932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.805943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.312526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.972074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.553136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.081636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.599923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.252562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.840690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.429677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.061847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.658923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.242459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.890989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.460530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.037175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.662191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.280647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.953790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.592652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.204322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.815347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.434857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.986884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.539819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.121402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.714264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.425633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.099614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.763630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.439485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.096347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.779964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.449038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.166178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.853243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.430078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.125238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.754142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.276808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.987273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.656369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.488653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.055662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.666643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.362645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.946259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.626657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.257174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.847319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.429302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.930639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.467670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.990259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.519418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.196876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.652216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.221449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.738108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.290284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.912477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.469487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.067289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.663593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.189464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.789681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.323450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.936514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.589615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.098697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.771261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.376622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.936424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.465676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.050180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.590648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.122212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.687951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.213595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.767224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.357220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.887079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.534931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.122950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.696740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.242180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.828794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.415678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.018671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.596643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.228836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.941224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.635645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.225789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.871192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.539178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.117389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.747549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.283231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.989819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.527949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.032838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.643748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.214427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.826910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.428565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.007419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.555335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.164476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.720553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.236579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.831885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.395350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.957622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.605439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.076718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.659838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.288867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.818045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.381939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.873458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.373288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.982829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.497466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.070368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.680815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.302530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.898231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.442463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.021477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.655609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.278573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.889494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.490330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.149579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.858035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.628331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.333758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.021163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.559417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.160406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.702570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.240815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.723330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.205600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.767396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.226085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.732483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.273821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.787992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.361747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.873205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.409886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.932475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.439505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.018694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.572299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.171359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.758769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.332707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.955065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.545102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.124839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.716745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.298699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.826036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.409290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.976921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.576713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.177066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.692825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.196745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.794046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.360083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.977055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.556068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.139656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.812216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.384110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.964739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.467519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.022347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.584718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.118204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.651196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.237774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.840770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.415224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.996713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.536544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.027200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.602918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.094117] LustreError: 164240:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 5786.098173] LustreError: 162664:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6000.608110] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6000.613120] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6000.620408] Lustre: Skipped 210 previous similar messages [ 6000.622616] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6002.550131] LustreError: 216492:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6002.641220] Lustre: server umount lustre-OST0000 complete [ 6006.752325] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6006.757083] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6007.776358] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6007.782606] Lustre: Skipped 1 previous similar message [ 6012.897587] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6012.901883] Lustre: Skipped 1 previous similar message [ 6014.948723] LustreError: 216693:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 6014.951362] LustreError: 216693:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6015.041111] Lustre: server umount lustre-MDT0000 complete [ 6016.693579] LustreError: 216115:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769758984 with bad export cookie 13191294449990866406 [ 6016.696795] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6016.700264] LustreError: 216115:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6016.778643] LustreError: 216896:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6016.781258] LustreError: 216896:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6016.847402] Lustre: server umount lustre-MDT0001 complete [ 6021.309816] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 6022.615649] Key type lgssc unregistered [ 6022.770672] LNet: 217428:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6022.773842] LNetError: 217428:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6022.788808] LNet: Removed LNI 192.168.203.157@tcp [ 6023.184273] Key type .llcrypt unregistered [ 6023.186128] Key type ._llcrypt unregistered [ 6034.342289] Key type ._llcrypt registered [ 6034.343729] Key type .llcrypt registered [ 6034.395871] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 6034.916896] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6034.943762] alg: No test for adler32 (adler32-zlib) [ 6035.838992] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 6035.942591] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 6037.527118] Key type lgssc registered [ 6038.295310] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6042.340506] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6042.358325] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6043.624629] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6045.390970] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6049.314807] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6049.513076] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6051.527865] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6054.033225] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6055.784932] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6058.938851] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6059.132893] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6061.698437] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6064.002448] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6069.356701] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 6073.315052] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6074.202989] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6074.216977] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6074.222422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.231567] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6074.232029] LustreError: 218820:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6075.872094] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6075.876625] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6075.881343] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6076.453338] Lustre: lustre-OST0000: Not available for connect from 192.168.203.57@tcp (stopping) [ 6079.969341] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6079.969671] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6081.198835] LustreError: 221195:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6081.252904] Lustre: server umount lustre-OST0000 complete [ 6094.327124] LustreError: 218820:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6110.484617] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6110.694641] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6113.292928] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6115.555619] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6118.300944] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 2 sec [ 6119.743271] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff932dc6dc8800.ost_server_uuid 50 [ 6119.906438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.913291] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6120.395309] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff932dc6dc8800.ost_server_uuid in FULL state after 0 sec [ 6125.025698] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6125.033573] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6125.037227] Lustre: Skipped 1 previous similar message [ 6128.048092] LustreError: 222512:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6128.053655] LustreError: 222512:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6128.107269] Lustre: server umount lustre-OST0000 complete [ 6130.145128] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6130.146622] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6130.149832] Lustre: Skipped 1 previous similar message [ 6135.941105] LustreError: 222714:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 6135.943803] LustreError: 222714:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6135.974141] Lustre: server umount lustre-MDT0000 complete [ 6136.288770] LustreError: 219522:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6136.298880] LustreError: 219522:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6137.482721] LustreError: 218781:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769759105 with bad export cookie 15295682035333155176 [ 6137.485509] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6137.487662] LustreError: 218781:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6137.626883] Lustre: server umount lustre-MDT0001 complete [ 6141.656179] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 6142.856715] Key type lgssc unregistered [ 6143.009386] LNet: 223447:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6143.013704] LNetError: 223447:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6143.021445] LNet: Removed LNI 192.168.203.157@tcp [ 6143.373229] Key type .llcrypt unregistered [ 6143.374449] Key type ._llcrypt unregistered [ 6156.385409] Key type ._llcrypt registered [ 6156.386895] Key type .llcrypt registered [ 6156.431660] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 6156.831065] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6156.866474] alg: No test for adler32 (adler32-zlib) [ 6157.801070] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 6157.916312] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 6159.527200] Key type lgssc registered [ 6159.991519] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6163.626861] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6163.634268] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6164.844292] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6166.434067] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6170.106067] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6170.319851] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6172.053574] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6173.349975] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6176.438070] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6176.607313] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6178.803691] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6182.351878] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6182.440360] Lustre: lustre-OST0001: new disk, initializing [ 6182.442373] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6182.501677] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6185.032829] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6189.674821] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 6190.633462] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6190.640948] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6190.701754] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6196.513382] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6202.651446] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6216.160741] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6216.168336] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6216.171794] Lustre: Skipped 1 previous similar message [ 6221.007870] LustreError: 228485:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 6221.054376] Lustre: server umount lustre-MDT0000 complete [ 6221.284594] LustreError: 225126:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6221.293066] LustreError: 225126:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 6222.691092] LustreError: 228208:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769759190 with bad export cookie 8861288545591548294 [ 6222.694168] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6222.696177] LustreError: 228208:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6222.776909] LustreError: 228686:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 6222.779637] LustreError: 228686:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6222.855153] Lustre: server umount lustre-MDT0001 complete [ 6234.613085] LustreError: 228888:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6234.615185] LustreError: 228888:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6234.636463] Lustre: server umount lustre-OST0000 complete [ 6245.500693] LustreError: 229089:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6245.503500] LustreError: 229089:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6245.559974] Lustre: server umount lustre-OST0001 complete [ 6254.701252] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 6259.031243] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6259.269614] LustreError: 230467:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6259.315965] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6260.926850] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6264.389657] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6266.292303] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6267.547803] Lustre: 231574:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6270.551074] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6270.705076] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6270.707545] Lustre: Skipped 1 previous similar message [ 6272.880433] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6274.791082] LustreError: 231929:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6276.423194] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6278.824694] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6283.751284] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49732 to 0x280000401:49761) [ 6288.024734] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6289.570451] Lustre: 233418:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6300.640825] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6300.645248] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6300.650123] Lustre: Skipped 3 previous similar messages [ 6300.652612] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6304.224974] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6304.229702] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6304.235511] Lustre: Skipped 2 previous similar messages [ 6305.408309] LustreError: 233725:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6305.410503] LustreError: 233725:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6305.439770] Lustre: server umount mylustre-MDT0000 complete [ 6306.869614] LustreError: 231193:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769759274 with bad export cookie 8861288545591551087 [ 6306.872484] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6306.873466] LustreError: 231193:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6306.999297] Lustre: server umount mylustre-MDT0001 complete [ 6318.582472] LustreError: 234126:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 6318.585598] LustreError: 234126:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 6318.609693] Lustre: server umount mylustre-OST0000 complete [ 6330.290523] Lustre: server umount mylustre-OST0001 complete [ 6338.487953] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 6342.227903] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6342.414942] LustreError: 235708:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6342.445186] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6342.447942] Lustre: Skipped 1 previous similar message [ 6343.951833] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6347.366289] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6349.002124] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6350.108577] Lustre: 236814:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6352.619853] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6352.672932] LustreError: 237170:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6354.546841] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49763 to 0x280000401:49793) [ 6354.879117] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6358.236492] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6360.555672] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6364.112103] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6365.508905] Lustre: 238655:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6373.344334] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6373.347637] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6373.352803] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 6373.355086] Lustre: Skipped 3 previous similar messages [ 6378.690395] LustreError: 238904:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6378.697252] LustreError: 238904:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 6378.732164] Lustre: server umount tfs-MDT0000 complete [ 6378.976122] LustreError: 235707:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6378.984623] LustreError: 235707:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 6380.114944] LustreError: 235684:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769759347 with bad export cookie 8861288545591552753 [ 6380.117403] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6380.119655] LustreError: 235684:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6380.243292] Lustre: server umount tfs-MDT0001 complete [ 6392.327365] Lustre: server umount tfs-OST0000 complete [ 6404.141190] Lustre: server umount tfs-OST0001 complete [ 6411.782334] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 6415.511908] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6415.681104] LustreError: 240885:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6415.708370] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6415.710412] Lustre: Skipped 3 previous similar messages [ 6417.183295] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6420.193934] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6421.744284] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6422.756448] Lustre: 241994:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6425.234062] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6426.405818] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49763 to 0x280000401:49825) [ 6427.613436] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6431.061979] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6433.404651] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6436.874233] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6438.160495] Lustre: 243835:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6445.465586] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 02:50:12 (1769759412) [ 6461.408304] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6461.412156] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6461.418072] Lustre: Skipped 3 previous similar messages [ 6461.420487] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6461.423369] Lustre: Skipped 3 previous similar messages [ 6463.489407] LustreError: 244203:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6463.493749] LustreError: 244203:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 6463.531869] Lustre: server umount lustre-MDT0000 complete [ 6464.886542] LustreError: 240859:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769759432 with bad export cookie 8861288545591554195 [ 6464.889306] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6464.895131] LustreError: 240859:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6467.039873] LustreError: 240879:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6467.046118] LustreError: 240879:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 6474.209232] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6474.212090] Lustre: Skipped 8 previous similar messages [ 6479.327153] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6479.418240] Lustre: server umount lustre-MDT0001 complete [ 6495.199112] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6495.238518] Lustre: server umount lustre-OST0000 complete [ 6502.691120] Lustre: server umount lustre-OST0001 complete [ 6505.015091] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_hostid [ 6508.435318] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 6521.282528] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 6521.282528] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 6521.282528] [ 6521.291889] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 6521.380374] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6521.393725] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6521.431859] Lustre: lustre-MDT0000: new disk, initializing [ 6521.459647] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6521.462805] Lustre: Skipped 3 previous similar messages [ 6521.469910] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6522.894041] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6527.099494] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6527.141833] Lustre: 246982:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6527.166704] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6527.168716] Lustre: Skipped 1 previous similar message [ 6527.201665] Lustre: lustre-MDT0001: new disk, initializing [ 6527.253516] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6527.257249] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6528.581470] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6532.924463] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6533.037163] Lustre: lustre-OST0000: new disk, initializing [ 6533.040061] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6535.098865] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6535.103985] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6535.126212] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6535.142919] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6539.550351] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6539.612567] Lustre: lustre-OST0001: new disk, initializing [ 6539.614539] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6541.249449] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6541.253528] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6541.285223] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6541.764832] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6546.197851] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 02:51:53 (1769759513) [ 6550.165874] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 02:51:57 (1769759517) [ 6561.762766] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6561.764911] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6561.767651] Lustre: Skipped 6 previous similar messages [ 6561.770078] Lustre: Skipped 2 previous similar messages [ 6567.726081] LustreError: 250393:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 6567.728992] LustreError: 250393:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 6567.748205] Lustre: server umount lustre-OST0000 complete [ 6572.001632] LustreError: 248874:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6572.008270] LustreError: 248874:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 6575.264231] Lustre: server umount lustre-MDT0000 complete [ 6576.711315] LustreError: 246972:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769759544 with bad export cookie 8861288545591555525 [ 6576.715860] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6576.716927] LustreError: 246972:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6582.891086] Lustre: server umount lustre-MDT0001 complete [ 6586.449169] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 6596.333960] Lustre: server umount lustre-OST0001 complete [ 6598.632702] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_hostid [ 6604.990087] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 6617.552215] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6617.641670] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6617.654259] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6617.687928] Lustre: lustre-MDT0000: new disk, initializing [ 6617.716271] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6619.008813] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6623.318240] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6623.363511] Lustre: 253842:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6623.464156] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6624.799747] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6628.001940] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6629.253586] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6631.589944] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6633.649413] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6633.723937] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6633.727967] Lustre: Skipped 1 previous similar message [ 6633.730121] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6633.771719] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6636.623613] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6639.072933] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6639.074109] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6639.076215] LustreError: Skipped 1 previous similar message [ 6639.080246] Lustre: Skipped 4 previous similar messages [ 6639.081048] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6639.085415] Lustre: Skipped 8 previous similar messages [ 6644.948883] Lustre: server umount lustre-OST0000 complete [ 6649.313129] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6652.381948] Lustre: server umount lustre-MDT0000 complete [ 6653.916305] LustreError: 253832:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769759621 with bad export cookie 8861288545591556995 [ 6653.919802] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6653.922907] LustreError: 253832:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6654.037304] Lustre: server umount lustre-MDT0001 complete [ 6657.732675] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 6658.751127] Key type lgssc unregistered [ 6658.881402] LNet: 257210:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6658.885217] LNetError: 257210:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6658.898318] LNet: Removed LNI 192.168.203.157@tcp [ 6659.257135] Key type .llcrypt unregistered [ 6659.258456] Key type ._llcrypt unregistered [ 6669.290620] Key type ._llcrypt registered [ 6669.291633] Key type .llcrypt registered [ 6669.327498] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 6669.702916] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6669.733471] alg: No test for adler32 (adler32-zlib) [ 6670.594890] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 6670.684945] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 6672.279139] Key type lgssc registered [ 6672.660728] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6675.207842] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6675.214862] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6676.375885] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6677.699425] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6680.434567] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6680.554830] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6681.831851] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6683.564889] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6684.690154] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6686.886049] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6687.004976] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6688.857787] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6690.631990] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6690.990986] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 6696.416443] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6696.418904] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6696.423921] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6697.706521] LustreError: 260913:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6697.740732] Lustre: server umount lustre-OST0000 complete [ 6698.967734] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6698.968926] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6698.973425] Lustre: Skipped 1 previous similar message [ 6698.977071] Lustre: Skipped 1 previous similar message [ 6704.096121] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6704.099056] Lustre: Skipped 1 previous similar message [ 6709.216087] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6709.218929] Lustre: Skipped 1 previous similar message [ 6713.311111] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6713.340206] LustreError: 261122:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 6713.341962] LustreError: 261122:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6713.364458] Lustre: server umount lustre-MDT0000 complete [ 6714.337040] LustreError: 258629:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6714.341550] LustreError: 258629:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6714.566468] LustreError: 258610:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769759682 with bad export cookie 16547250135469208021 [ 6714.569142] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6714.571128] LustreError: 258610:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6714.630962] LustreError: 261326:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6714.633089] LustreError: 261326:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6714.686491] Lustre: server umount lustre-MDT0001 complete [ 6718.079506] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 6719.067829] Key type lgssc unregistered [ 6719.202409] LNet: 261857:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6719.205193] LNetError: 261857:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6719.212315] LNet: Removed LNI 192.168.203.157@tcp [ 6719.517054] Key type .llcrypt unregistered [ 6719.518059] Key type ._llcrypt unregistered [ 6725.997538] Key type ._llcrypt registered [ 6725.999073] Key type .llcrypt registered [ 6726.035243] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 6726.298126] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6726.317891] alg: No test for adler32 (adler32-zlib) [ 6727.164576] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 6727.248345] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 6728.823195] Key type lgssc registered [ 6729.201939] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6732.175097] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6732.183121] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6747.615270] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 6752.735168] LustreError: 263008:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.157@tcp: failed processing log, type 4: rc = -110 [ 6779.359217] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6781.349539] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6783.208881] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6783.980323] LustreError: 263670:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 6784.018338] Lustre: server umount lustre-OST0000 complete [ 6789.745040] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 6790.827740] Key type lgssc unregistered [ 6790.958342] LNet: 264505:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6790.961257] LNetError: 264505:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6790.967301] LNet: Removed LNI 192.168.203.157@tcp [ 6791.274627] Key type .llcrypt unregistered [ 6791.276140] Key type ._llcrypt unregistered [ 6805.437957] Key type ._llcrypt registered [ 6805.439077] Key type .llcrypt registered [ 6805.481827] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 6805.859777] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6805.883584] alg: No test for adler32 (adler32-zlib) [ 6806.751715] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 6806.847930] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 6808.431136] Key type lgssc registered [ 6808.826833] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6813.387687] loop0: detected capacity change from 0 to 204800000 [ 6813.444542] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6813.448510] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6813.570421] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6813.934482] loop0: detected capacity change from 0 to 204800000 [ 6813.989964] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6813.993603] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6814.119139] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6814.485420] loop0: detected capacity change from 0 to 204800000 [ 6814.550624] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6814.554352] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6814.594351] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6814.960286] loop0: detected capacity change from 0 to 204800000 [ 6815.064122] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6815.461054] loop0: detected capacity change from 0 to 204800000 [ 6815.471692] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6816.771739] loop0: detected capacity change from 0 to 204800000 [ 6816.782431] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6818.127051] loop0: detected capacity change from 0 to 204800000 [ 6818.134990] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6819.411265] loop0: detected capacity change from 0 to 204800000 [ 6819.424606] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6820.696990] loop0: detected capacity change from 0 to 204800000 [ 6820.725535] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6820.734358] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6820.767193] Lustre: 267874:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6820.770351] Lustre: 267874:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 6820.774670] Lustre: 267874:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 6820.778107] Lustre: 267874:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6820.781577] Lustre: 267874:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6820.784865] Lustre: 267874:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 6820.788550] Lustre: 267874:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6820.791885] Lustre: 267874:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 6820.795254] Lustre: 267874:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 6820.798754] Lustre: 267874:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 6820.802189] Lustre: 267874:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 6820.805547] Lustre: 267874:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 6820.808690] Lustre: 267874:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 6820.811668] Lustre: 267874:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 6823.309741] LustreError: 268138:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6823.350925] Lustre: server umount MGS complete [ 6823.667164] loop0: detected capacity change from 0 to 204800000 [ 6823.693230] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6823.855217] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6823.872551] Lustre: lustre-MDT0000: Aborting client recovery [ 6824.190341] loop1: detected capacity change from 0 to 204800000 [ 6824.222845] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6824.262059] Lustre: 268305:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6824.265618] Lustre: 268305:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6824.269762] Lustre: 268305:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 6824.274695] Lustre: 268305:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 6824.278179] Lustre: 268305:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6824.281724] Lustre: 268305:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 6824.285940] Lustre: 268305:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6824.292676] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6824.358452] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6824.385249] Lustre: lustre-MDT0001: Aborting client recovery [ 6824.701872] loop2: detected capacity change from 0 to 204800000 [ 6824.730290] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6824.755099] Lustre: 268385:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6824.759247] Lustre: 268385:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6824.762822] Lustre: 268385:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6824.766694] Lustre: 268385:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6824.772483] Lustre: 268385:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6824.778065] Lustre: 268385:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6824.782832] Lustre: 268385:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6824.830121] Lustre: Found index 0 for lustre-OST0000, updating log [ 6824.870399] Lustre: lustre-OST0000: Aborting recovery [ 6825.198207] loop3: detected capacity change from 0 to 204800000 [ 6825.230467] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6825.257900] Lustre: 268470:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6825.262451] Lustre: 268470:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6825.266537] Lustre: 268470:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6825.270499] Lustre: 268470:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6825.274587] Lustre: 268470:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6825.278460] Lustre: 268470:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6825.283100] Lustre: 268470:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6830.569070] LustreError: 268249:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 6833.633834] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6833.633834] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6833.639208] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 6833.646399] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 6834.660814] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 6834.661209] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 6836.738377] LustreError: 268217:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 213us: rc = -2 [ 6850.017164] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6850.017242] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6850.024541] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6850.030117] Lustre: Skipped 3 previous similar messages [ 6855.137117] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6855.140601] Lustre: Skipped 3 previous similar messages [ 6860.259535] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6860.261568] Lustre: Skipped 3 previous similar messages [ 6862.303125] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6862.336434] LustreError: 268823:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6862.338346] LustreError: 268823:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 6862.371102] Lustre: server umount lustre-MDT0000 complete [ 6862.673718] LustreError: 268311:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769759830 with bad export cookie 4310706283609830077 [ 6862.675202] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6862.677101] LustreError: 268311:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6865.375499] LustreError: 268484:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6865.375758] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6865.375963] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6865.381657] LustreError: 268484:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6865.385162] Lustre: Skipped 4 previous similar messages [ 6867.423418] LustreError: 268484:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6867.428456] LustreError: 268484:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6868.860677] LustreError: 268876:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 6868.863513] LustreError: 268876:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6868.927148] Lustre: server umount lustre-MDT0001 complete [ 6875.307073] LustreError: 268927:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 6875.309721] LustreError: 268927:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6875.331836] Lustre: server umount lustre-OST0000 complete [ 6881.708282] LustreError: 268978:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6881.710115] LustreError: 268978:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6881.756477] Lustre: server umount lustre-OST0001 complete [ 6884.211235] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 02:57:31 (1769759851) [ 6893.337918] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_hostid [ 6896.359192] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 6908.166525] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6908.256710] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6908.270302] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6908.306346] Lustre: lustre-MDT0000: new disk, initializing [ 6908.332548] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6908.334879] Lustre: Skipped 2 previous similar messages [ 6908.340519] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6909.632633] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6913.674609] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6913.713819] Lustre: 272508:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6913.737076] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6913.738923] Lustre: Skipped 1 previous similar message [ 6913.773942] Lustre: lustre-MDT0001: new disk, initializing [ 6913.806705] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6913.819973] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6913.823763] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6915.043710] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6918.063596] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6919.243174] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6921.530118] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6921.624568] Lustre: lustre-OST0000: new disk, initializing [ 6921.626827] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6921.655693] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6922.300454] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6922.305248] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6922.335344] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6923.642586] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 6926.858682] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6929.253158] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 6932.975501] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 6941.289100] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 6949.820902] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 6999.009372] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6999.009571] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6999.013709] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6999.016579] Lustre: Skipped 4 previous similar messages [ 7001.646068] LustreError: 276114:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 7001.648288] LustreError: 276114:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7001.681974] Lustre: server umount lustre-OST0000 complete [ 7004.128309] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7004.131753] Lustre: Skipped 2 previous similar messages [ 7008.978148] Lustre: server umount lustre-MDT0000 complete [ 7009.247931] LustreError: 272519:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7009.253343] LustreError: 272519:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7010.104283] LustreError: 273286:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769759977 with bad export cookie 4310706283609832786 [ 7010.107653] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7010.107886] LustreError: 273286:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7010.218661] Lustre: server umount lustre-MDT0001 complete [ 7012.845292] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7014.070151] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7015.637269] Lustre: server umount MGS complete [ 7019.273061] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7019.412517] LustreError: 278078:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7019.420097] LustreError: 278090:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 7019.446534] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7020.687286] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7023.347915] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7024.592612] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7026.161824] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7027.243705] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7029.331468] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7031.138176] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7032.853048] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7054.306490] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7054.310808] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7054.311095] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7054.313124] LustreError: Skipped 1 previous similar message [ 7054.315380] Lustre: Skipped 2 previous similar messages [ 7058.479625] LustreError: 280726:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7058.481418] LustreError: 280726:0:(obd_class.h:479:obd_check_dev()) Skipped 19 previous similar messages [ 7058.517313] Lustre: server umount lustre-OST0000 complete [ 7064.544627] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7064.544824] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7064.548487] Lustre: Skipped 2 previous similar messages [ 7065.815251] Lustre: server umount lustre-MDT0000 complete [ 7067.116132] LustreError: 278057:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769760034 with bad export cookie 4310706283609834585 [ 7067.118963] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7067.121213] LustreError: 278057:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7067.228744] Lustre: server umount lustre-MDT0001 complete [ 7070.621625] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 7071.637496] Key type lgssc unregistered [ 7071.762357] LNet: 281660:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7071.764650] LNetError: 281660:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7071.771412] LNet: Removed LNI 192.168.203.157@tcp [ 7072.035456] Key type .llcrypt unregistered [ 7072.037328] Key type ._llcrypt unregistered [ 7083.173521] Key type ._llcrypt registered [ 7083.174447] Key type .llcrypt registered [ 7083.211603] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_hostid [ 7088.926512] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 7089.204540] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7089.244560] alg: No test for adler32 (adler32-zlib) [ 7090.083737] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 7090.157957] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 7091.727182] Key type lgssc registered [ 7092.071427] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7102.729569] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7102.734762] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7103.821958] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7103.832174] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7103.866257] Lustre: lustre-MDT0000: new disk, initializing [ 7103.889688] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7103.895739] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7105.089591] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7109.074810] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7109.131224] Lustre: 285561:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7109.150437] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7109.151858] Lustre: Skipped 1 previous similar message [ 7109.185615] Lustre: lustre-MDT0001: new disk, initializing [ 7109.216468] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7109.228808] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7109.232407] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7110.465443] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7113.507322] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7114.651804] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7116.814670] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7116.909798] Lustre: lustre-OST0000: new disk, initializing [ 7116.911787] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7116.937215] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7118.716996] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7121.695931] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7125.493333] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7125.497773] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7125.524696] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7130.207822] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 7136.493272] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 7142.256701] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 7146.708798] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 7186.914412] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7186.914848] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7186.917697] LustreError: Skipped 1 previous similar message [ 7186.923039] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7186.927414] Lustre: Skipped 1 previous similar message [ 7191.468985] LustreError: 289167:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 7191.502930] Lustre: server umount lustre-OST0000 complete [ 7196.127876] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7196.130637] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7196.135127] Lustre: Skipped 1 previous similar message [ 7196.137132] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7197.155162] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7197.155176] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7202.271930] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7202.273678] Lustre: Skipped 2 previous similar messages [ 7206.368287] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7206.370608] Lustre: Skipped 1 previous similar message [ 7206.879176] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7206.917662] LustreError: 289368:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 7206.922741] LustreError: 289368:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7206.954358] Lustre: server umount lustre-MDT0000 complete [ 7208.185084] LustreError: 288735:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769760175 with bad export cookie 9091690106413956006 [ 7208.188856] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7208.188980] LustreError: 288735:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7208.254849] LustreError: 289570:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7208.256839] LustreError: 289570:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7208.312321] Lustre: server umount lustre-MDT0001 complete [ 7211.162842] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7212.440608] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7214.106644] LustreError: 290509:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 7214.109332] LustreError: 290509:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 7214.130888] Lustre: server umount MGS complete [ 7218.004736] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7218.164565] LustreError: 291142:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 7218.188481] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7219.468426] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7222.373817] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7222.499434] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7223.756847] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7225.524896] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7226.775465] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7229.028772] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7230.901285] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7232.634693] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7268.323064] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7268.323198] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7268.324104] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7268.328531] Lustre: Skipped 1 previous similar message [ 7273.006092] LustreError: 293805:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7273.009053] LustreError: 293805:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 7273.048127] Lustre: server umount lustre-OST0000 complete [ 7278.560375] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7278.565581] Lustre: Skipped 1 previous similar message [ 7280.410426] Lustre: server umount lustre-MDT0000 complete [ 7281.657066] LustreError: 291109:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769760249 with bad export cookie 9091690106413957833 [ 7281.659347] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7281.660735] LustreError: 291109:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7281.726319] LustreError: 294207:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7281.728135] LustreError: 294207:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 7281.775462] Lustre: server umount lustre-MDT0001 complete [ 7285.361796] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 7286.409183] Key type lgssc unregistered [ 7286.539426] LNet: 294738:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7286.542177] LNetError: 294738:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7286.551324] LNet: Removed LNI 192.168.203.157@tcp [ 7286.867554] Key type .llcrypt unregistered [ 7286.868746] Key type ._llcrypt unregistered [ 7293.539078] Key type ._llcrypt registered [ 7293.539993] Key type .llcrypt registered [ 7293.573886] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_hostid [ 7299.448872] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 7299.789402] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7299.869100] alg: No test for adler32 (adler32-zlib) [ 7300.731506] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 7300.811764] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 7302.391136] Key type lgssc registered [ 7302.760628] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7310.267831] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 7315.065057] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7315.071950] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7316.168560] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7316.181436] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7316.220494] Lustre: lustre-MDT0000: new disk, initializing [ 7316.248897] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7316.257486] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7317.543750] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7321.676179] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7321.777312] Lustre: lustre-OST0000: new disk, initializing [ 7321.779386] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7321.781964] Lustre: Skipped 1 previous similar message [ 7321.805133] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7322.885194] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 7322.889296] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 7322.900311] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 7323.802914] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7630.304182] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7630.314134] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7631.426964] LustreError: 299654:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 7631.524697] Lustre: server umount lustre-MDT0000 complete [ 7632.959139] LustreError: 299567:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769760600 with bad export cookie 4822300057755300657 [ 7632.960980] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7632.964380] LustreError: 299567:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7634.988846] LustreError: 299856:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 7634.990789] LustreError: 299856:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7635.032578] Lustre: server umount lustre-OST0000 complete [ 7644.086593] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_hostid [ 7647.765623] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 7660.101018] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7660.200955] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7660.216182] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7660.254728] Lustre: lustre-MDT0000: new disk, initializing [ 7660.291468] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7660.299217] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7661.673328] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7665.940529] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7665.985022] Lustre: 302881:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7666.009825] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7666.012449] Lustre: Skipped 1 previous similar message [ 7666.045218] Lustre: lustre-MDT0001: new disk, initializing [ 7666.084366] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7666.097165] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7666.100222] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7667.342715] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7670.374054] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7671.535636] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7673.783231] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7673.884694] Lustre: lustre-OST0000: new disk, initializing [ 7673.887103] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7673.934206] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7675.803256] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7678.874884] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7680.350911] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7681.017652] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7681.021021] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7681.049834] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7681.459950] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 7682.993324] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 7683.075856] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 7686.114765] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7686.115113] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7686.119902] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7686.122299] LustreError: Skipped 1 previous similar message [ 7686.125906] Lustre: Skipped 1 previous similar message [ 7689.901269] LustreError: 305820:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 7689.904336] LustreError: 305820:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7689.952549] Lustre: server umount lustre-OST0000 complete [ 7691.743731] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7691.746240] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7691.750016] Lustre: Skipped 1 previous similar message [ 7691.752131] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7696.352115] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7696.352480] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7697.406189] LustreError: 306021:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 7697.408186] LustreError: 306021:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7697.438775] Lustre: server umount lustre-MDT0000 complete [ 7698.682090] LustreError: 303673:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769760666 with bad export cookie 4822300057758241434 [ 7698.684499] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7698.686140] LustreError: 303673:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7698.804568] Lustre: server umount lustre-MDT0001 complete [ 7699.996690] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 7700.617120] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 03:11:07 (1769760667) [ 7702.941691] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7703.122071] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7704.491514] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7707.615601] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7709.163965] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7711.088135] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7712.477541] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7714.878035] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7717.128216] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7719.211371] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7721.812544] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7721.892407] Lustre: lustre-OST0001: new disk, initializing [ 7721.896274] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7721.937473] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7721.940948] Lustre: Skipped 2 previous similar messages [ 7722.232859] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7722.237793] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7722.260320] LustreError: 309336:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 7723.297716] LustreError: 309336:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 7723.306234] LustreError: 309336:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 7724.325532] LustreError: 309345:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 7724.329861] LustreError: 309345:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 7724.377950] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7726.371912] LustreError: 309345:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 7726.376244] LustreError: 309345:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 7728.316462] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7728.385994] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 7729.749964] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff932dc4117000.ost_server_uuid 50 [ 7730.355393] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff932dc4117000.ost_server_uuid in FULL state after 0 sec [ 7730.466874] LustreError: 309336:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 7730.471768] LustreError: 309336:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 7731.828525] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff932dc4117000.ost_server_uuid 50 [ 7732.441804] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff932dc4117000.ost_server_uuid in FULL state after 0 sec [ 7733.548583] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7747.610208] Lustre: lustre-OST0001: Not available for connect from 192.168.203.57@tcp (stopping) [ 7747.613327] Lustre: Skipped 1 previous similar message [ 7748.070855] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7748.074784] Lustre: Skipped 1 previous similar message [ 7750.509082] LustreError: 310593:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 7750.511261] LustreError: 310593:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 7750.531472] Lustre: server umount lustre-OST0001 complete [ 7753.185754] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7753.189121] LustreError: 308389:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7753.198198] LustreError: 308389:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7758.307361] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7758.307803] LustreError: 308389:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7758.310040] Lustre: Skipped 5 previous similar messages [ 7758.315861] LustreError: 308389:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7758.702823] LustreError: 310796:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7758.705915] LustreError: 310796:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7758.747415] Lustre: server umount lustre-OST0000 complete [ 7763.425078] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7763.430693] Lustre: Skipped 2 previous similar messages [ 7766.237062] Lustre: server umount lustre-MDT0000 complete [ 7767.550583] LustreError: 306777:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769760735 with bad export cookie 4822300057758242316 [ 7767.551492] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7767.555723] LustreError: 306777:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7767.671261] Lustre: server umount lustre-MDT0001 complete [ 7771.489248] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 7772.684224] Key type lgssc unregistered [ 7772.830368] LNet: 311730:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7772.834322] LNetError: 311730:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7772.843438] LNet: Removed LNI 192.168.203.157@tcp [ 7773.181225] Key type .llcrypt unregistered [ 7773.182929] Key type ._llcrypt unregistered [ 7782.819989] Key type ._llcrypt registered [ 7782.821308] Key type .llcrypt registered [ 7782.861805] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 7783.329181] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7783.347434] alg: No test for adler32 (adler32-zlib) [ 7784.203887] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 7784.290064] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 7785.871096] Key type lgssc registered [ 7786.232945] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7788.892084] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7788.899991] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7790.079685] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7791.358417] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7793.083811] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7795.517511] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7795.679435] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7797.093037] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7800.336189] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7800.449297] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7802.407273] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7804.290166] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7805.604089] LustreError: 314656:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7805.610664] LustreError: 314656:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7806.562577] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7806.637615] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7808.525293] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7810.226137] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 7814.632977] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7814.633889] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 7817.022168] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 7817.105882] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 7895.519839] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7895.522079] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7895.527321] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7896.545316] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7896.547450] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7896.548408] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7901.667194] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7901.668706] Lustre: Skipped 2 previous similar messages [ 7906.785149] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7906.788055] Lustre: Skipped 1 previous similar message [ 7907.295105] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7907.321176] LustreError: 316517:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 7907.339626] Lustre: server umount lustre-OST0001 complete [ 7911.905930] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7911.907714] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7911.907977] LustreError: 314657:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7911.910547] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7911.913465] Lustre: Skipped 1 previous similar message [ 7917.025349] LustreError: 315376:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7917.030524] LustreError: 315376:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7922.146562] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7922.148520] LustreError: 316099:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7922.149725] Lustre: Skipped 4 previous similar messages [ 7922.155463] LustreError: 316099:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7923.167121] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7923.186595] LustreError: 316721:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 7923.189185] LustreError: 316721:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7923.225855] Lustre: server umount lustre-OST0000 complete [ 7927.264816] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7927.269365] Lustre: Skipped 1 previous similar message [ 7930.558311] LustreError: 316923:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 7930.560279] LustreError: 316923:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7930.582755] Lustre: server umount lustre-MDT0000 complete [ 7931.820992] LustreError: 316418:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769760899 with bad export cookie 12163407888445900615 [ 7931.823954] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7931.824993] LustreError: 316418:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7931.938050] Lustre: server umount lustre-MDT0001 complete [ 7935.331551] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 7936.300961] Key type lgssc unregistered [ 7936.418385] LNet: 317656:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7936.420667] LNetError: 317656:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7936.428260] LNet: Removed LNI 192.168.203.157@tcp [ 7936.708293] Key type .llcrypt unregistered [ 7936.709811] Key type ._llcrypt unregistered [ 7954.702840] Key type ._llcrypt registered [ 7954.704252] Key type .llcrypt registered [ 7954.744209] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 7955.236729] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7955.290590] alg: No test for adler32 (adler32-zlib) [ 7956.144168] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 7956.221734] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 7957.807174] Key type lgssc registered [ 7958.241711] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7961.490862] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7961.497116] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7962.650387] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7964.063192] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7966.971043] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7967.094083] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7968.420953] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7969.431297] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7971.691519] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7971.813718] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7972.834306] LustreError: 321556:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7972.837892] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 7972.840441] LustreError: 321556:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7973.755230] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7976.689494] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7976.790380] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7977.840939] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 7978.818229] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 7982.841466] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7987.543440] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7997.919844] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7997.924054] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7997.931897] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7998.435397] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7998.438912] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7998.439405] Lustre: Skipped 1 previous similar message [ 7998.442449] Lustre: Skipped 1 previous similar message [ 8002.176412] LustreError: 323320:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 8002.203934] Lustre: server umount lustre-MDT0000 complete [ 8003.520092] LustreError: 319898:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769760971 with bad export cookie 10869128751002866811 [ 8003.523970] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8003.525369] LustreError: 319898:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8003.552140] LustreError: 322339:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8003.553145] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8003.553792] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8003.557733] LustreError: 322339:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8003.561858] Lustre: Skipped 2 previous similar messages [ 8003.592732] LustreError: 323521:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 8003.594481] LustreError: 323521:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8003.650179] Lustre: server umount lustre-MDT0001 complete [ 8010.991090] LustreError: 323721:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 8010.993657] LustreError: 323721:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8011.010815] Lustre: server umount lustre-OST0000 complete [ 8012.259379] Lustre: server umount lustre-OST0001 complete [ 8017.584219] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 8021.229227] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8021.370395] LustreError: 325096:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8021.393754] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8022.787354] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 8025.751088] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8027.262626] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 8028.282492] Lustre: 326201:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8030.608682] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8030.729100] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8030.730797] Lustre: Skipped 1 previous similar message [ 8031.780415] LustreError: 326556:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8031.783113] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 8032.766434] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 8035.875291] LustreError: 326556:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8035.881730] LustreError: 326556:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8036.041539] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8037.682886] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 8038.166874] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 8041.368044] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8042.501651] Lustre: 328042:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8051.682497] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8051.682826] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8051.684487] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8051.684494] Lustre: Skipped 1 previous similar message [ 8051.693901] Lustre: Skipped 3 previous similar messages [ 8054.523956] LustreError: 328146:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 8054.526825] LustreError: 328146:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 8054.555442] Lustre: server umount lustre-MDT0000 complete [ 8055.710640] LustreError: 325077:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769761023 with bad export cookie 10869128751002868470 [ 8055.713072] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8055.715450] LustreError: 325077:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8055.836253] Lustre: server umount lustre-MDT0001 complete [ 8067.257088] LustreError: 328550:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 8067.258807] LustreError: 328550:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 8067.275444] Lustre: server umount lustre-OST0000 complete [ 8078.709913] LustreError: 328752:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 8078.711524] LustreError: 328752:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8078.749746] Lustre: server umount lustre-OST0001 complete [ 8079.866526] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 8080.330302] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 8080.846468] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 03:17:27 (1769761047) [ 8088.083661] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 8091.000907] loop0: detected capacity change from 0 to 1099511627776 [ 8091.143270] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 8091.145930] *** VALIDATE xfs *** [ 8091.151033] XFS (loop0): Mounting V5 Filesystem [ 8091.158707] XFS (loop0): Ending clean mount [ 8091.551092] loop1: detected capacity change from 0 to 18691697672192 [ 8111.569304] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_hostid [ 8114.498555] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 8125.694782] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8125.774131] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8125.783840] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8125.815984] Lustre: lustre-MDT0000: new disk, initializing [ 8125.838248] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8125.840025] Lustre: Skipped 1 previous similar message [ 8125.844922] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8127.085831] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 8130.974858] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8131.018492] Lustre: 333105:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8131.043470] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8131.044983] Lustre: Skipped 1 previous similar message [ 8131.077045] Lustre: lustre-MDT0001: new disk, initializing [ 8131.123548] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8131.128382] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8132.314820] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 8135.219334] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8136.334778] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8138.489395] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8138.589988] Lustre: lustre-OST0000: new disk, initializing [ 8138.592127] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8140.347460] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8140.350666] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8140.389708] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8140.443622] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 8143.442149] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8144.737299] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 8144.806083] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 8146.044602] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 8146.116554] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 8150.498287] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8150.498479] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8150.501364] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8150.504177] Lustre: Skipped 1 previous similar message [ 8150.507403] Lustre: Skipped 4 previous similar messages [ 8160.737779] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8160.740033] Lustre: Skipped 3 previous similar messages [ 8161.247117] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8161.267417] LustreError: 336034:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 8161.269432] LustreError: 336034:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8161.303712] Lustre: server umount lustre-OST0000 complete [ 8165.856653] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8165.860603] Lustre: Skipped 1 previous similar message [ 8168.662808] Lustre: server umount lustre-MDT0000 complete [ 8169.870070] LustreError: 333883:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769761137 with bad export cookie 10869128751002870024 [ 8169.872997] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8169.873753] LustreError: 333883:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8169.983535] Lustre: server umount lustre-MDT0001 complete [ 8172.523861] XFS (loop0): Unmounting Filesystem [ 8174.107196] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 03:19:01 (1769761141) [ 8176.295500] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8176.443091] LustreError: 337254:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8176.465945] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8176.468598] Lustre: Skipped 2 previous similar messages [ 8177.762304] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 8180.478926] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8181.837416] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 8183.555308] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8184.701188] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8186.852082] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8188.780232] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 8190.471350] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8211.935846] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8211.938811] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8211.942800] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8211.944456] Lustre: Skipped 2 previous similar messages [ 8215.469097] LustreError: 339613:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 8215.470877] LustreError: 339613:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 8215.507835] Lustre: server umount lustre-OST0000 complete [ 8230.879924] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8230.937028] Lustre: server umount lustre-MDT0000 complete [ 8232.208288] LustreError: 337234:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769761199 with bad export cookie 10869128751002870906 [ 8232.211156] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8232.214512] LustreError: 337234:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8232.323373] Lustre: server umount lustre-MDT0001 complete [ 8235.931324] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 8236.972206] Key type lgssc unregistered [ 8237.101253] LNet: 340548:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8237.104890] LNetError: 340548:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8237.115509] LNet: Removed LNI 192.168.203.157@tcp [ 8237.417155] Key type .llcrypt unregistered [ 8237.418285] Key type ._llcrypt unregistered [ 8247.523903] Key type ._llcrypt registered [ 8247.525365] Key type .llcrypt registered [ 8247.567140] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 8247.880407] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8247.900935] alg: No test for adler32 (adler32-zlib) [ 8248.755240] Lustre: Lustre: Build Version: 2.17.50_72_g285a242 [ 8248.845927] LNet: Added LNI 192.168.203.157@tcp [8/256/0/180] [ 8250.423143] Key type lgssc registered [ 8250.825430] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8253.389356] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8253.397820] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8254.576864] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8255.915776] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 8258.717178] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8258.848206] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8260.137399] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 8261.773964] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8262.909751] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8264.963083] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8265.063070] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8266.751550] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 8268.389566] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8274.401701] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8274.401836] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 8274.402589] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8274.406556] Lustre: Skipped 1 previous similar message [ 8274.971570] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.57@tcp (stopping) [ 8274.975180] Lustre: Skipped 1 previous similar message [ 8277.889505] LustreError: 344361:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 8277.909252] Lustre: server umount lustre-MDT0001 complete [ 8279.520628] LustreError: 342780:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8279.528046] LustreError: 342780:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8280.083866] LustreError: 344223:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.57@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8281.536727] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8281.572506] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 8281.574261] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8281.588519] Lustre: 342060:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8281.710197] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8283.045190] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 8287.206279] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 8291.814221] LustreError: 345581:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 8291.817582] LustreError: 345581:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 8291.821813] LustreError: 342062:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 8291.840580] LustreError: 345584:0:(obd_config.c:2075:class_config_llog_handler()) MGC192.168.203.157@tcp: cfg command failed: rc = -17 [ 8291.843384] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 8291.843384] [ 8587.288753] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.57@tcp (stopping) [ 8589.280811] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 8589.281036] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8592.190610] LustreError: 357020:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 8592.192396] LustreError: 357020:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8592.213920] Lustre: server umount lustre-MDT0001 complete [ 8592.402367] LustreError: 344223:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.57@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8594.402410] LustreError: 342066:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8594.407488] LustreError: 342066:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8595.531855] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8595.571221] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 8595.573758] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8595.588889] Lustre: 342060:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8595.711981] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8597.128443] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 8601.061853] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 8603.109859] LustreError: 358140:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 8603.112572] LustreError: 358140:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 8603.115722] LustreError: 342062:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 8603.118120] LustreError: 342062:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 8603.136044] LustreError: 358143:0:(obd_config.c:776:class_setup()) Device 8 already setup (type osp) [ 8603.139286] LustreError: 358143:0:(obd_config.c:873:class_detach()) OBD device 8 still set up [ 8603.141679] LustreError: 358143:0:(obd_config.c:2075:class_config_llog_handler()) MGC192.168.203.157@tcp: cfg command failed: rc = -17 [ 8603.144764] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.157@tcp [ 8603.144764] [ 8903.136907] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 8903.138459] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8903.139482] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8903.139486] Lustre: Skipped 2 previous similar messages [ 8903.146567] Lustre: Skipped 2 previous similar messages [ 8906.426983] LustreError: 369828:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 8906.428753] LustreError: 369828:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8906.449581] Lustre: server umount lustre-MDT0001 complete [ 8908.257107] LustreError: 342070:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8908.264993] LustreError: 342070:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8909.583033] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8909.609883] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 8909.611986] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8909.623211] Lustre: 342059:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8909.723425] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 8910.889865] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 8914.916762] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 8919.011788] LustreError: 371046:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 8919.013973] LustreError: 371046:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 8919.016062] LustreError: 342062:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 8919.018365] LustreError: 342062:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 8919.032167] LustreError: 371049:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 8919.036940] LustreError: 371049:0:(obd_config.c:2075:class_config_llog_handler()) MGC192.168.203.157@tcp: cfg command failed: rc = -114 [ 8919.039688] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 8919.039688] [ 9227.234419] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9227.234859] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9227.235166] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9227.235534] Lustre: Skipped 2 previous similar messages [ 9227.248914] Lustre: Skipped 1 previous similar message [ 9230.267033] LustreError: 382752:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 9230.268733] LustreError: 382752:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 9230.296937] Lustre: server umount lustre-MDT0000 complete [ 9231.425744] LustreError: 342051:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769762199 with bad export cookie 7692998607541957749 [ 9231.430078] LustreError: 342051:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9231.430293] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9232.351709] LustreError: 344223:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9232.351985] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9232.358823] Lustre: Skipped 1 previous similar message [ 9236.384241] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9236.386125] Lustre: Skipped 3 previous similar messages [ 9237.562384] LustreError: 382953:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 9237.564382] LustreError: 382953:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 9237.609365] Lustre: server umount lustre-MDT0001 complete [ 9238.935099] Lustre: server umount lustre-OST0000 complete [ 9241.445924] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_hostid [ 9244.077959] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 9254.840059] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 03:37:01 (1769762221) [ 9256.823876] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9256.901847] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9256.912248] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9256.944103] Lustre: lustre-MDT0000: new disk, initializing [ 9256.965842] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9256.971884] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9258.140717] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 9261.902856] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9261.936916] Lustre: 385673:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9261.958473] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9261.959949] Lustre: Skipped 1 previous similar message [ 9261.991525] Lustre: lustre-MDT0001: new disk, initializing [ 9262.020527] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9262.031224] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9262.034363] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9263.143270] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 9266.252419] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9267.452995] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9269.622940] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9269.714467] Lustre: lustre-OST0000: new disk, initializing [ 9269.716754] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9269.745037] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9271.421826] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_default_debug -1 all [ 9274.339754] Lustre: DEBUG MARKER: oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9274.359692] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9274.362774] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9274.393156] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9277.407827] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9277.409771] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9277.413515] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9282.812919] LustreError: 388111:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 9282.815331] LustreError: 388111:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 9282.844152] Lustre: server umount lustre-MDT0000 complete [ 9283.975179] LustreError: 387415:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769762251 with bad export cookie 7692998607541959688 [ 9283.978405] LustreError: 386466:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 9283.978673] LustreError: MGC192.168.203.157@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9283.980260] LustreError: 387415:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9284.085039] Lustre: server umount lustre-MDT0001 complete [ 9294.490233] Lustre: server umount lustre-OST0000 complete [ 9302.878636] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing set_hostid [ 9305.661925] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing load_modules_local [ 9314.778834] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 9006 sec ========== 03:38:01 (1769762281) [ 9315.300383] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 03:38:02 (1769762282) === [ 9316.313902] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 03:38:03 (1769762283) === [ 9328.270079] Lustre: DEBUG MARKER: oleg357-server.virtnet: executing unload_modules_local [ 9329.174258] Key type lgssc unregistered [ 9329.294293] LNet: 393617:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9329.296365] LNetError: 393617:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9329.307305] LNet: Removed LNI 192.168.203.157@tcp [ 9329.572282] Key type .llcrypt unregistered [ 9329.573034] Key type ._llcrypt unregistered