[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 446146425 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2448MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.002338] x2apic enabled [ 0.003005] Switched APIC routing to physical x2apic. [ 0.004008] kvm-guest: setup PV IPIs [ 0.006832] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008013] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009007] pid_max: default: 32768 minimum: 301 [ 0.010126] LSM: Security Framework initializing [ 0.011037] Yama: becoming mindful. [ 0.012026] SELinux: Initializing. [ 0.014003] *** VALIDATE selinux *** [ 0.022239] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027327] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029026] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030085] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031087] *** VALIDATE tmpfs *** [ 0.032448] *** VALIDATE proc *** [ 0.034194] *** VALIDATE cgroup *** [ 0.035005] *** VALIDATE cgroup2 *** [ 0.036245] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037130] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038003] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039023] Spectre V2 : User space: Vulnerable [ 0.040005] Speculative Store Bypass: Vulnerable [ 0.043580] debug: unmapping init [mem 0xffffffff9e459000-0xffffffff9e460fff] [ 0.045152] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.046580] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.047013] ... version: 2 [ 0.048008] ... bit width: 48 [ 0.049010] ... generic registers: 4 [ 0.050010] ... value mask: 0000ffffffffffff [ 0.051010] ... max period: 00007fffffffffff [ 0.052007] ... fixed-purpose events: 3 [ 0.053008] ... event mask: 000000070000000f [ 0.055212] rcu: Hierarchical SRCU implementation. [ 0.057637] smp: Bringing up secondary CPUs ... [ 0.058594] x86: Booting SMP configuration: [ 0.059015] .... node #0, CPUs: #1 #2 #3 [ 0.062673] smp: Brought up 1 node, 4 CPUs [ 0.064009] smpboot: Max logical packages: 1 [ 0.065012] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.256024] node 0 deferred pages initialised in 190ms [ 0.259142] devtmpfs: initialized [ 0.260192] x86/mm: Memory block size: 128MB [ 0.263234] gcov: version magic: 0x41383552 [ 0.265324] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.269065] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.272326] pinctrl core: initialized pinctrl subsystem [ 0.274134] [ 0.274756] ************************************************************* [ 0.277017] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.279013] ** ** [ 0.282015] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.285014] ** ** [ 0.287014] ** This means that this kernel is built to expose internal ** [ 0.290013] ** IOMMU data structures, which may compromise security on ** [ 0.292011] ** your system. ** [ 0.295012] ** ** [ 0.297020] ** If you see this message and you are not debugging the ** [ 0.300008] ** kernel, report this immediately to your vendor! ** [ 0.302007] ** ** [ 0.305010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.307008] ************************************************************* [ 0.310740] NET: Registered protocol family 16 [ 0.312427] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.315041] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.318045] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.322131] cpuidle: using governor menu [ 0.323535] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.326503] PCI: Using configuration type 1 for base access [ 0.329151] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.337056] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.339013] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.341129] cryptd: max_cpu_qlen set to 1000 [ 0.344322] ACPI: Added _OSI(Module Device) [ 0.345012] ACPI: Added _OSI(Processor Device) [ 0.347009] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.349010] ACPI: Added _OSI(Processor Aggregator Device) [ 0.354332] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.360386] ACPI: Interpreter enabled [ 0.362050] ACPI: PM: (supports S0 S3 S4 S5) [ 0.363025] ACPI: Using IOAPIC for interrupt routing [ 0.365087] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.369398] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.380380] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.383028] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.386012] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.390101] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.395438] acpiphp: Slot [2] registered [ 0.397101] acpiphp: Slot [5] registered [ 0.398091] acpiphp: Slot [6] registered [ 0.400076] acpiphp: Slot [7] registered [ 0.401110] acpiphp: Slot [8] registered [ 0.403080] acpiphp: Slot [9] registered [ 0.405078] acpiphp: Slot [10] registered [ 0.406097] acpiphp: Slot [3] registered [ 0.408071] acpiphp: Slot [4] registered [ 0.409064] acpiphp: Slot [11] registered [ 0.411091] acpiphp: Slot [12] registered [ 0.412066] acpiphp: Slot [13] registered [ 0.414078] acpiphp: Slot [14] registered [ 0.415073] acpiphp: Slot [15] registered [ 0.417075] acpiphp: Slot [16] registered [ 0.418066] acpiphp: Slot [17] registered [ 0.420073] acpiphp: Slot [18] registered [ 0.421122] acpiphp: Slot [19] registered [ 0.423071] acpiphp: Slot [20] registered [ 0.424055] acpiphp: Slot [21] registered [ 0.425069] acpiphp: Slot [22] registered [ 0.426055] acpiphp: Slot [23] registered [ 0.427066] acpiphp: Slot [24] registered [ 0.428052] acpiphp: Slot [25] registered [ 0.428992] acpiphp: Slot [26] registered [ 0.430061] acpiphp: Slot [27] registered [ 0.431061] acpiphp: Slot [28] registered [ 0.431948] acpiphp: Slot [29] registered [ 0.433050] acpiphp: Slot [30] registered [ 0.434085] acpiphp: Slot [31] registered [ 0.435071] PCI host bridge to bus 0000:00 [ 0.436014] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.439010] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.441013] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.444010] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.445009] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.447011] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.448141] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.450735] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.453318] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.461014] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.464046] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.467011] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.469010] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.471011] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.473776] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.477949] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.480031] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.483948] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.489012] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.501014] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.505012] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.512000] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.519016] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.527017] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.541022] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.555179] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.561010] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.566009] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.580012] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.589748] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.596015] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.603023] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.620019] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.630177] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.634010] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.638011] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.653017] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.661952] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.669017] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.674016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.689023] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.697419] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.702015] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.706014] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.719017] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.728087] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.730286] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.732257] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.734346] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.736150] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.740744] iommu: Default domain type: Passthrough [ 0.742458] SCSI subsystem initialized [ 0.744137] ACPI: bus type USB registered [ 0.745083] usbcore: registered new interface driver usbfs [ 0.747060] usbcore: registered new interface driver hub [ 0.748070] usbcore: registered new device driver usb [ 0.750181] pps_core: LinuxPPS API ver. 1 registered [ 0.752008] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.756067] PTP clock support registered [ 0.758185] EDAC MC: Ver: 3.0.0 [ 0.760160] PCI: Using ACPI for IRQ routing [ 0.764047] NetLabel: Initializing [ 0.765013] NetLabel: domain hash size = 128 [ 0.767010] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.768088] NetLabel: unlabeled traffic allowed by default [ 0.771098] vgaarb: loaded [ 0.772337] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.773010] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.777392] clocksource: Switched to clocksource kvm-clock [ 0.887486] VFS: Disk quotas dquot_6.6.0 [ 0.888787] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.891259] *** VALIDATE ramfs *** [ 0.892490] *** VALIDATE hugetlbfs *** [ 0.893926] pnp: PnP ACPI init [ 0.896217] pnp: PnP ACPI: found 6 devices [ 0.911516] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.914294] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.916116] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.917893] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.919931] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.922054] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.925218] NET: Registered protocol family 2 [ 0.927493] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.931310] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.934985] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.938477] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.940664] TCP: Hash tables configured (established 65536 bind 65536) [ 0.943093] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.945297] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.947636] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.950411] NET: Registered protocol family 1 [ 0.953189] RPC: Registered named UNIX socket transport module. [ 0.956680] RPC: Registered udp transport module. [ 0.959108] RPC: Registered tcp transport module. [ 0.961481] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.963914] NET: Registered protocol family 44 [ 0.965879] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.967590] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.969646] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.971778] PCI: CLS 0 bytes, default 64 [ 0.973549] Unpacking initramfs... [ 2.347392] debug: unmapping init [mem 0xffff94d37cc54000-0xffff94d37ffbffff] [ 2.353852] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.355879] software IO TLB: mapped [mem 0x00000000b8c54000-0x00000000bcc54000] (64MB) [ 2.360187] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.819723] Initialise system trusted keyrings [ 2.821041] Key type blacklist registered [ 2.822502] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.831037] zbud: loaded [ 2.833675] *** VALIDATE nfs *** [ 2.834666] *** VALIDATE nfs4 *** [ 2.835862] pstore: using deflate compression [ 2.840595] Platform Keyring initialized [ 2.931853] NET: Registered protocol family 38 [ 2.934484] Key type asymmetric registered [ 2.936417] Asymmetric key parser 'x509' registered [ 2.939355] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.943981] io scheduler mq-deadline registered [ 2.946274] io scheduler kyber registered [ 2.948456] io scheduler bfq registered [ 2.950918] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.954930] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.958949] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.963048] ACPI: Power Button [PWRF] [ 2.969425] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.976819] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.995381] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.005884] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.027331] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.057773] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.086572] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.092322] Non-volatile memory driver v1.3 [ 3.093572] Linux agpgart interface v0.103 [ 3.126043] virtio_blk virtio1: [vda] 145928 512-byte logical blocks (74.7 MB/71.3 MiB) [ 3.130175] vda: detected capacity change from 0 to 74715136 [ 3.145408] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.148544] vdb: detected capacity change from 0 to 1073741824 [ 3.164925] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.167819] vdc: detected capacity change from 0 to 2621440000 [ 3.192335] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.195044] vdd: detected capacity change from 0 to 2621440000 [ 3.215232] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.220188] vde: detected capacity change from 0 to 4294967296 [ 3.238061] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.240707] vdf: detected capacity change from 0 to 4294967296 [ 3.247825] libphy: Fixed MDIO Bus: probed [ 3.261271] usbcore: registered new interface driver usbserial_generic [ 3.263470] usbserial: USB Serial support registered for generic [ 3.266084] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.270537] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.272198] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.275270] mousedev: PS/2 mouse device common for all mice [ 3.278493] rtc_cmos 00:05: RTC can wake from S4 [ 3.283238] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.283647] rtc_cmos 00:05: registered as rtc0 [ 3.290640] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.293614] intel_pstate: CPU model not supported [ 3.297603] hid: raw HID events driver (C) Jiri Kosina [ 3.298376] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.302263] usbcore: registered new interface driver usbhid [ 3.309052] usbhid: USB HID core driver [ 3.312090] drop_monitor: Initializing network drop monitor service [ 3.313148] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.317117] Initializing XFRM netlink socket [ 3.323244] NET: Registered protocol family 10 [ 3.328241] Segment Routing with IPv6 [ 3.330443] NET: Registered protocol family 17 [ 3.333913] mpls_gso: MPLS GSO support [ 3.343753] RAS: Correctable Errors collector initialized. [ 3.346389] AVX version of gcm_enc/dec engaged. [ 3.347429] AES CTR mode by8 optimization enabled [ 3.434940] sched_clock: Marking stable (3434869054, 0)->(4240314845, -805445791) [ 3.439545] registered taskstats version 1 [ 3.441376] Loading compiled-in X.509 certificates [ 3.443682] zswap: loaded using pool lzo/zbud [ 3.485399] Key type big_key registered [ 3.504725] Key type encrypted registered [ 3.507469] ima: No TPM chip found, activating TPM-bypass! [ 3.510625] ima: Allocated hash algorithm: sha1 [ 3.512939] ima: No architecture policies found [ 3.515577] evm: Initialising EVM extended attributes: [ 3.518260] evm: security.selinux [ 3.519718] evm: security.ima [ 3.521506] evm: security.capability [ 3.523602] evm: HMAC attrs: 0x1 [ 3.528482] rtc_cmos 00:05: setting system clock to 2026-08-12 13:33:55 UTC (1786541635) [ 3.536635] debug: unmapping init [mem 0xffffffff9f403000-0xffffffff9f5fffff] [ 3.540078] debug: unmapping init [mem 0xffffffff9e182000-0xffffffff9e458fff] [ 3.554104] Write protecting the kernel read-only data: 28672k [ 3.558664] debug: unmapping init [mem 0xffffffff9c803000-0xffffffff9c9fffff] [ 3.561495] debug: unmapping init [mem 0xffffffff9d114000-0xffffffff9d1fffff] [ 3.595924] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.604610] systemd[1]: Detected virtualization kvm. [ 3.606962] systemd[1]: Detected architecture x86-64. [ 3.611880] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.639400] systemd[1]: No hostname configured. [ 3.644567] systemd[1]: Set hostname to . [ 3.646400] random: systemd: uninitialized urandom read (16 bytes read) [ 3.648390] systemd[1]: Initializing machine ID from random generator. [ 3.700603] random: ln: uninitialized urandom read (6 bytes read) [ 3.786801] random: systemd: uninitialized urandom read (16 bytes read) [ 3.790873] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 3.804630] systemd[1]: Starting Setup Virtual Console... Starting Setup Virtual Console... [ 3.811982] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Swap. Starting Apply Kernel Variables... [ OK ] Reached target Slices. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Timers. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Control Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.580764] device-mapper: uevent: version 1.0.3 [ 4.583203] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev[ 5.706095] virtio_net virtio0 ens2: renamed from eth0 Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.893793] random: fast init done [ 7.880164] scsi host0: ata_piix [ 7.961233] scsi host1: ata_piix [ 7.964735] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 7.970056] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 11.061751] random: crng init done [ 11.066145] random: 7 urandom warning(s) missed due to ratelimiting [ 11.410045] hrtimer: interrupt took 2094141 ns [ 14.620846] dracut-initqueue[589]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 15.777965] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 18.102638] printk: systemd: 26 output lines suppressed due to ratelimiting [ 18.672974] SELinux: Disabled at runtime. [ 18.773390] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 18.799363] systemd[1]: Detected virtualization kvm. [ 18.808663] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 19.765198] systemd[1]: initrd-switch-root.service: Succeeded. [ 19.771381] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 19.778958] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 19.783894] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 19.792575] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 19.800317] systemd[1]: Starting Journal Service... Starting Journal Service... [ 19.811609] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. Mounting Kernel Debug File System... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. Starting Apply Kernel Variables... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Created slice system-sshd\x2dkeygen.slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on Process Core Dump Socket. [ OK ] Stopped target Initrd File Systems. Mounting POSIX Message Queue File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. Starting Create list of required st…ce nodes for the current kernel... Starting Remount Root and Kernel File Systems... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target rpc_pipefs.target. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. Mounting Huge Pages Fil[ 20.073631] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS e System... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-getty.slice. Starting udev Coldplug all Devices... [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 20.524541] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 21.074350] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 21.080959] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 21.286256] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 21.404381] EDAC sbridge: Ver: 1.1.2 [ 25.323953] Key type dns_resolver registered [ 25.876183] NFS: Registering the id_resolver key type [ 25.881466] Key type id_resolver registered [ 25.883637] Key type id_legacy registered [* ] A start job is running for Configur…-only root support (6s / no limit) [** ] A start job is running for Configur…-only root support (6s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Rebuild Dynamic Linker Cache... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. Starting Login Service... [ OK ] Started dnf makecache --timer. [ OK ] Started D-Bus System Message Bus. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting System Logging Service... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg338-server login: [ 54.868802] libcfs: loading out-of-tree module taints kernel. [ 54.886681] Key type ._llcrypt registered [ 54.887953] Key type .llcrypt registered [ 54.928240] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_hostid [ 61.931420] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 62.523627] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 62.529557] alg: No test for adler32 (adler32-zlib) [ 63.515054] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 63.792717] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 65.408122] Key type lgssc registered [ 65.888847] Lustre: Echo OBD driver; http://www.lustre.org/ [ 71.982785] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 86.170333] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 90.651868] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 90.659239] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 91.744764] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 91.756912] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 91.797869] Lustre: lustre-MDT0000: new disk, initializing [ 91.822366] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 91.829498] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 93.134222] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 98.296118] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 98.338447] Lustre: 6480:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 98.351775] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 98.353598] Lustre: Skipped 1 previous similar message [ 98.393094] Lustre: lustre-MDT0001: new disk, initializing [ 98.417852] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 98.425859] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 98.429637] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 99.851606] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 102.136695] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 105.417140] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 105.505087] Lustre: lustre-OST0000: new disk, initializing [ 105.506659] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 105.509328] Lustre: 8419:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 105.526149] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 107.445611] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 108.139083] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 108.141798] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 108.157857] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 112.684446] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 112.734752] Lustre: lustre-OST0001: new disk, initializing [ 112.736632] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 112.739903] Lustre: 9494:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 112.759525] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 113.587298] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 113.591645] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 113.609268] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 114.806441] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 120.708698] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 125.260569] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 127.222140] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing check_logdir /tmp/testlogs/ [ 128.853902] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing yml_node [ 130.397109] Lustre: DEBUG MARKER: Client: 2.17.57.1 [ 131.245781] Lustre: DEBUG MARKER: MDS: 2.17.57.1 [ 132.131359] Lustre: DEBUG MARKER: OSS: 2.17.57.1 [ 132.712415] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Aug 12 09:36:04 EDT 2026 [ 138.342669] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 138.894222] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 154.593534] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 154.593808] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 154.597830] Lustre: Skipped 1 previous similar message [ 156.827312] Lustre: server umount lustre-MDT0000 complete [ 158.054605] LustreError: 6474:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786541790 with bad export cookie 17635836175779219675 [ 158.056165] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 158.058904] LustreError: 6474:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 159.714925] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 159.715205] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 159.719940] Lustre: Skipped 3 previous similar messages [ 159.722562] Lustre: Skipped 3 previous similar messages [ 163.744508] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 163.747289] Lustre: Skipped 1 previous similar message [ 164.257419] Lustre: server umount lustre-MDT0001 complete [ 171.645835] Lustre: server umount lustre-OST0000 complete [ 187.360154] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 187.409296] Lustre: server umount lustre-OST0001 complete [ 189.597429] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_hostid [ 192.825377] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 205.331504] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 205.422768] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 205.436405] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 205.476989] Lustre: lustre-MDT0000: new disk, initializing [ 205.500744] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 205.508251] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 206.963890] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 211.488962] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 211.548991] Lustre: 15035:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 211.675346] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 213.116874] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 217.046344] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 219.087385] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 221.518611] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 221.627780] Lustre: lustre-OST0000: new disk, initializing [ 221.629365] Lustre: Skipped 1 previous similar message [ 221.631855] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 221.634829] Lustre: Skipped 2 previous similar messages [ 221.637809] Lustre: 17002:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 221.669976] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 221.673181] Lustre: Skipped 1 previous similar message [ 223.489709] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 223.493582] Lustre: Skipped 1 previous similar message [ 223.495977] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 223.534949] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 223.792842] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 227.828498] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 229.363664] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 229.439933] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 230.929142] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 231.016592] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 233.957103] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 233.957560] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 233.963607] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 233.965705] LustreError: Skipped 1 previous similar message [ 233.968593] Lustre: Skipped 1 previous similar message [ 239.074958] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 239.077494] Lustre: Skipped 1 previous similar message [ 246.240130] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 246.303494] Lustre: server umount lustre-OST0000 complete [ 247.776980] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 247.781205] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 247.785791] Lustre: Skipped 1 previous similar message [ 247.789207] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 247.792846] Lustre: Skipped 2 previous similar messages [ 253.852465] Lustre: server umount lustre-MDT0000 complete [ 254.433176] LustreError: 15847:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 254.440884] LustreError: 15847:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 255.230586] LustreError: 15830:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786541887 with bad export cookie 17635836175779221194 [ 255.234157] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 255.238152] LustreError: 15830:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 255.375259] Lustre: server umount lustre-MDT0001 complete [ 257.262513] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 09:38:08 (1786541888) [ 259.805594] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 260.021957] LustreError: 19264:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 260.062986] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 261.686732] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 264.939223] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 266.676068] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 269.422432] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 271.619562] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 274.203148] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 276.580154] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 279.430586] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 291.720754] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 295.905813] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 295.907731] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 295.909841] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 295.914188] Lustre: Skipped 3 previous similar messages [ 295.917190] Lustre: Skipped 3 previous similar messages [ 301.091892] Lustre: server umount lustre-MDT0000 complete [ 302.619565] LustreError: 19246:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786541934 with bad export cookie 17635836175779222076 [ 302.622668] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 302.625435] LustreError: 19246:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 302.752057] Lustre: server umount lustre-MDT0001 complete [ 314.526935] Lustre: server umount lustre-OST0000 complete [ 321.937251] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 325.899677] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 326.038657] LustreError: 23662:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 326.055461] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 326.058754] Lustre: Skipped 2 previous similar messages [ 327.450595] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 330.567576] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 332.139385] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 333.198737] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 335.549305] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 337.808932] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 340.937581] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 341.002661] Lustre: lustre-OST0001: new disk, initializing [ 341.005340] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 341.008913] Lustre: 25976:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 342.347882] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 342.352946] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 342.381650] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 343.091221] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 347.661253] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 350.294535] Lustre: Setting parameter general.obdfilter.*.client_cache_count=256 in log params [ 350.297332] Lustre: Skipped 1 previous similar message [ 356.322043] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 356.324787] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 356.330456] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 356.332761] Lustre: Skipped 3 previous similar messages [ 359.335989] Lustre: server umount lustre-MDT0000 complete [ 360.619097] LustreError: 23644:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786541992 with bad export cookie 17635836175779222916 [ 360.622622] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 360.624221] LustreError: 23644:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 360.742532] Lustre: server umount lustre-MDT0001 complete [ 371.136995] Lustre: server umount lustre-OST0000 complete [ 382.681159] Lustre: server umount lustre-OST0001 complete [ 387.857770] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 391.481311] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 391.618763] LustreError: 28983:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 391.637294] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 391.639271] Lustre: Skipped 3 previous similar messages [ 392.974838] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 395.724993] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 397.208046] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 398.164555] Lustre: 30131:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 400.306137] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 401.445245] LustreError: 30483:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 402.259989] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 404.951504] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 406.842788] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 409.814072] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 410.940451] Lustre: 32007:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 416.226900] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 416.227144] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 416.231513] Lustre: Skipped 4 previous similar messages [ 418.588275] Lustre: server umount lustre-MDT0000 complete [ 419.773962] LustreError: 32009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786542051 with bad export cookie 17635836175779224155 [ 419.778248] LustreError: 32009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 419.778272] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 419.888695] Lustre: server umount lustre-MDT0001 complete [ 431.557765] Lustre: server umount lustre-OST0000 complete [ 443.033765] Lustre: server umount lustre-OST0001 complete [ 445.945839] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 09:41:17 (1786542077) [ 454.592722] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 457.848935] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 457.977236] LustreError: 34800:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 459.410342] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 462.179631] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 463.610373] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 464.553128] Lustre: 35949:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 466.650293] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 468.620858] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 471.439915] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 473.482038] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 476.392816] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 477.515883] Lustre: 37827:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 482.786817] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 482.787945] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 482.788969] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 482.788974] Lustre: Skipped 8 previous similar messages [ 482.791211] Lustre: Skipped 4 previous similar messages [ 484.819428] Lustre: server umount lustre-MDT0000 complete [ 485.995061] LustreError: 34783:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786542117 with bad export cookie 17635836175779225240 [ 485.998068] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 485.998520] LustreError: 34783:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 486.107267] Lustre: server umount lustre-MDT0001 complete [ 497.090487] Lustre: server umount lustre-OST0000 complete [ 508.506995] Lustre: server umount lustre-OST0001 complete [ 511.320337] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 09:42:23 (1786542143) [ 516.029487] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 519.340562] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 519.479358] LustreError: 39962:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 519.484610] LustreError: 39962:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 520.721026] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 523.427241] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 523.557321] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 523.559620] Lustre: Skipped 8 previous similar messages [ 524.768463] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 525.685182] Lustre: 41110:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 527.797442] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 529.823089] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 532.665414] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 534.529723] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 537.511130] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 538.958477] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 542.099060] Lustre: server umount lustre-MDT0000 complete [ 543.291183] LustreError: 40711:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786542175 with bad export cookie 17635836175779226325 [ 543.294959] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 549.484440] Lustre: server umount lustre-MDT0001 complete [ 552.183673] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 553.248150] LustreError: 43840:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 553.253170] LustreError: 43840:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@ffff94d3fda20a80 x1873324744660608/t0(0) o250->MGC192.168.203.138@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1786542185 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'ptlrpcd_00_00.0' uid:0 gid:0 projid:4294967295 [ 553.263992] LustreError: 43840:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 553.376681] LustreError: 3619:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff94d3d02bed80 x1873324744662272/t0(0) o250->MGC192.168.203.138@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 553.494203] LustreError: 41473:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 553.501980] LustreError: 41473:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 554.885876] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 557.728730] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 559.236682] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 561.563087] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 572.515731] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 10 sec [ 573.284048] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 574.304246] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 574.820983] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 578.534815] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 578.536571] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 578.540895] Lustre: Skipped 6 previous similar messages [ 582.227229] Lustre: server umount lustre-MDT0000 complete [ 583.397229] LustreError: 39945:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786542215 with bad export cookie 17635836175779227403 [ 583.401484] LustreError: 39945:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 583.402775] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 583.522878] Lustre: server umount lustre-MDT0001 complete [ 595.012321] Lustre: server umount lustre-OST0000 complete [ 606.553159] Lustre: server umount lustre-OST0001 complete [ 609.253820] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 09:44:01 (1786542241) [ 614.277587] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 617.754155] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 617.876617] LustreError: 47602:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 619.183583] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 622.025094] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 623.559292] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 624.531587] Lustre: 48754:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 624.535752] Lustre: 48754:0:(mgs_llog.c:1450:mgs_modify_param()) Skipped 1 previous similar message [ 626.641018] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 628.601775] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 631.255751] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 633.067063] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 635.986576] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 637.293662] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 642.533981] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 642.535269] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 642.539180] Lustre: Skipped 15 previous similar messages [ 646.610722] Lustre: server umount lustre-MDT0000 complete [ 647.752178] LustreError: 50635:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786542279 with bad export cookie 17635836175779228229 [ 647.754735] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 647.756850] LustreError: 50635:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 647.881030] Lustre: server umount lustre-MDT0001 complete [ 659.968242] Lustre: server umount lustre-OST0000 complete [ 671.382508] Lustre: server umount lustre-OST0001 complete [ 674.510501] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 09:45:06 (1786542306) [ 675.278021] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 675.805322] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 09:45:07 (1786542307) [ 679.062729] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_hostid [ 681.747515] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 688.508061] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 688.585307] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 688.615860] Lustre: lustre-MDT0000: new disk, initializing [ 688.636720] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 689.826490] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 693.261635] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 695.282941] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 695.370409] Lustre: 54819:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 696.893521] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 696.901930] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 697.113810] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 700.570713] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 701.203491] Lustre: DEBUG MARKER: create test files [ 712.161717] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 712.163942] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 712.167231] Lustre: Skipped 4 previous similar messages [ 717.302567] Lustre: server umount lustre-OST0000 complete [ 718.493927] Lustre: server umount lustre-MDT0000 complete [ 721.949801] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 722.842603] Key type lgssc unregistered [ 722.957630] LNet: 56432:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 722.960617] LNetError: 56432:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 722.967260] LNet: Removed LNI 192.168.203.138@tcp [ 723.282073] Key type .llcrypt unregistered [ 723.283307] Key type ._llcrypt unregistered [ 732.984070] Key type ._llcrypt registered [ 732.985282] Key type .llcrypt registered [ 733.019771] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 733.306841] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 733.337100] alg: No test for adler32 (adler32-zlib) [ 734.184637] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 734.275950] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 735.848159] Key type lgssc registered [ 736.176022] Lustre: Echo OBD driver; http://www.lustre.org/ [ 738.598844] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 738.607098] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 739.743889] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 741.032410] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 743.408554] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 745.614123] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 745.739929] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 747.515588] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 749.773019] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 749.860314] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 749.881704] Lustre: 59137:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 749.884687] Lustre: 59137:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 749.887444] Lustre: 59137:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 749.890028] Lustre: 59137:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 749.893029] Lustre: 59137:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 749.895546] Lustre: 59137:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 752.697568] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 754.104436] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 760.289689] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 760.289817] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 760.297638] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 761.934551] Lustre: server umount lustre-OST0000 complete [ 763.180639] Lustre: server umount lustre-MDT0000 complete [ 766.551451] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 767.524224] Key type lgssc unregistered [ 767.658352] LNet: 60750:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 767.661334] LNetError: 60750:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 767.669284] LNet: Removed LNI 192.168.203.138@tcp [ 767.970965] Key type .llcrypt unregistered [ 767.972455] Key type ._llcrypt unregistered [ 778.723181] Key type ._llcrypt registered [ 778.724370] Key type .llcrypt registered [ 778.762547] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 779.162923] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 779.183774] alg: No test for adler32 (adler32-zlib) [ 780.030515] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 780.109499] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 781.680125] Key type lgssc registered [ 781.999759] Lustre: Echo OBD driver; http://www.lustre.org/ [ 784.344987] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 784.352715] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 785.476411] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 786.716281] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 789.008763] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 791.161655] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 791.264410] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 793.037593] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 795.261691] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 795.365675] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 795.386980] Lustre: 63557:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 795.390359] Lustre: 63557:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 795.392630] Lustre: 63557:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 795.394758] Lustre: 63557:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 795.397209] Lustre: 63557:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 795.399652] Lustre: 63557:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 798.222858] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 800.737425] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 800.737505] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 800.746347] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 802.440630] Lustre: server umount lustre-OST0000 complete [ 809.767758] Lustre: server umount lustre-MDT0000 complete [ 812.993814] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 813.907666] Key type lgssc unregistered [ 814.031258] LNet: 65117:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 814.033705] LNetError: 65117:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 814.042219] LNet: Removed LNI 192.168.203.138@tcp [ 814.313686] Key type .llcrypt unregistered [ 814.314785] Key type ._llcrypt unregistered [ 819.653978] Key type ._llcrypt registered [ 819.655615] Key type .llcrypt registered [ 819.694095] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_hostid [ 825.602272] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 825.955064] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 825.960356] alg: No test for adler32 (adler32-zlib) [ 826.795013] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 826.871950] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 828.448115] Key type lgssc registered [ 828.768283] Lustre: Echo OBD driver; http://www.lustre.org/ [ 839.136172] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 839.141141] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 840.219043] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 840.230451] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 840.267043] Lustre: lustre-MDT0000: new disk, initializing [ 840.287919] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 840.294677] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 841.520186] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 845.524375] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 845.563060] Lustre: 68243:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 845.583830] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 845.585790] Lustre: Skipped 1 previous similar message [ 845.628203] Lustre: lustre-MDT0001: new disk, initializing [ 845.664909] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 845.676688] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 845.680321] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 846.957194] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 850.474805] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 852.162648] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 854.226328] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 854.323023] Lustre: lustre-OST0000: new disk, initializing [ 854.324894] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 854.327417] Lustre: 70208:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 854.353390] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 856.280202] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 856.950391] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 856.953832] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 856.978736] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 859.843377] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 861.196435] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 861.271839] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 862.499236] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 862.558529] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 867.298415] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 867.298470] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 867.301367] LustreError: Skipped 1 previous similar message [ 867.301931] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 867.305180] Lustre: Skipped 1 previous similar message [ 867.310186] Lustre: Skipped 1 previous similar message [ 869.266933] Lustre: server umount lustre-OST0000 complete [ 871.392795] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 871.395545] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 871.400362] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 872.417522] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 872.417931] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 876.696841] Lustre: server umount lustre-MDT0000 complete [ 877.862263] LustreError: 70207:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786542509 with bad export cookie 6309426792740385821 [ 877.864949] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 877.865346] LustreError: 70207:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 877.975615] Lustre: server umount lustre-MDT0001 complete [ 880.569624] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 09:48:32 (1786542512) [ 883.679018] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 895.451775] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_hostid [ 898.250549] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 909.456895] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 909.530906] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 909.540991] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 909.578193] Lustre: lustre-MDT0000: new disk, initializing [ 909.598641] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 909.605747] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 910.870680] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 914.917092] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 914.959875] Lustre: 76214:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 914.988489] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 914.990741] Lustre: Skipped 1 previous similar message [ 915.023450] Lustre: lustre-MDT0001: new disk, initializing [ 915.057774] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 915.069761] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 915.074180] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 916.317837] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 919.776166] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 921.464226] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 923.457772] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 923.556585] Lustre: lustre-OST0000: new disk, initializing [ 923.558488] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 923.561688] Lustre: 78181:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 923.585188] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 925.238113] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 926.262959] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 926.266622] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 926.294522] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 928.798101] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 930.178083] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 930.249974] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 931.659523] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 931.734683] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 936.417743] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 936.418638] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 936.421020] LustreError: Skipped 1 previous similar message [ 936.426665] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 936.429889] Lustre: Skipped 2 previous similar messages [ 938.450929] Lustre: server umount lustre-OST0000 complete [ 941.024860] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 941.026786] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 941.029743] Lustre: Skipped 1 previous similar message [ 941.031510] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 951.776951] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 951.779069] Lustre: Skipped 4 previous similar messages [ 954.336088] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 954.397277] Lustre: server umount lustre-MDT0000 complete [ 955.573127] LustreError: 77007:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786542587 with bad export cookie 6309426792740386689 [ 955.575602] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 955.576393] LustreError: 77007:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 955.677807] Lustre: server umount lustre-MDT0001 complete [ 958.303935] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 09:49:50 (1786542590) [ 960.348015] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 960.480443] LustreError: 80588:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 960.486041] LustreError: 80588:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 960.501403] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 961.714548] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 964.396466] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 965.721176] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 967.954550] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 969.583595] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 971.642383] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 973.454279] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 975.626848] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 976.781793] Lustre: 83258:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting b12e0501-f02c-49b9-9178-c65d8a7c2a6b at adminstrative request [ 979.940413] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 979.943787] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 989.153478] Lustre: MGS: Client b12e0501-f02c-49b9-9178-c65d8a7c2a6b (at 0@lo) reconnecting [ 989.157124] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x578f963d3d9f7ae5 to 0x578f963d3d9f7d70 [ 989.160892] Lustre: MGC192.168.203.138@tcp: Connection restored to 0@lo (at 0@lo) [ 1009.156983] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1009.226248] Lustre: lustre-OST0001: new disk, initializing [ 1009.227835] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1009.230260] Lustre: 83653:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1009.259795] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1009.262860] Lustre: Skipped 2 previous similar messages [ 1011.114450] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1014.839346] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1015.882784] Lustre: server umount lustre-OST0001 complete [ 1018.864337] LustreError: 82408:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1018.865457] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1018.867126] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1018.874622] Lustre: Skipped 2 previous similar messages [ 1023.180518] Lustre: server umount lustre-OST0000 complete [ 1026.016778] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1030.553514] Lustre: server umount lustre-MDT0000 complete [ 1031.843497] LustreError: 82429:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786542663 with bad export cookie 6309426792740388208 [ 1031.848680] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1031.852950] LustreError: 82429:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1031.974063] Lustre: server umount lustre-MDT0001 complete [ 1034.872281] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 09:51:06 (1786542666) [ 1035.438319] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1036.059527] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 09:51:07 (1786542667) [ 1036.635980] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1037.211672] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 09:51:09 (1786542669) [ 1037.754714] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1038.301746] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 09:51:10 (1786542670) [ 1038.798202] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1039.376632] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 09:51:11 (1786542671) [ 1039.952914] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1041.957797] loop: module loaded [ 1041.961823] loop0: detected capacity change from 0 to 4294967296 [ 1041.975092] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1042.466231] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1043.565764] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1044.894818] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1053.711741] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1053.724587] LustreError: 87762:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1053.727730] LustreError: 87762:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1053.731059] LustreError: 87762:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1053.733770] LustreError: 87762:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1059.610429] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_hostid [ 1063.049277] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 1074.789943] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1074.871129] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1074.881843] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1074.925915] Lustre: lustre-MDT0000: new disk, initializing [ 1074.946418] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1074.953501] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1076.311582] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1080.443135] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1080.481198] Lustre: 90619:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1080.572088] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1081.832308] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1085.571926] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1087.454331] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1089.689594] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1089.799345] Lustre: 92587:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1091.788352] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1095.633785] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1097.107611] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1099.255220] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1099.286770] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1100.236377] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 3 sec [ 1101.596391] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1101.676473] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1104.355962] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1104.356414] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1104.357344] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1104.357349] Lustre: Skipped 4 previous similar messages [ 1104.360780] Lustre: Skipped 3 previous similar messages [ 1104.363329] LustreError: Skipped 1 previous similar message [ 1108.430711] Lustre: server umount lustre-OST0000 complete [ 1115.805424] Lustre: server umount lustre-MDT0000 complete [ 1117.051305] LustreError: 90657:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786542749 with bad export cookie 6309426792740388789 [ 1117.054191] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1117.056375] LustreError: 90657:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1117.168042] Lustre: server umount lustre-MDT0001 complete [ 1119.903147] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 09:52:31 (1786542751) [ 1122.215480] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1122.368564] LustreError: 95013:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1122.376505] LustreError: 95013:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 1123.771841] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1126.709146] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1128.214302] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1130.554444] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1132.345719] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1134.642063] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1136.668126] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1139.126598] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1140.564282] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1147.763914] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 7 sec [ 1149.068707] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1149.138880] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1151.312840] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1151.374077] Lustre: lustre-OST0001: new disk, initializing [ 1151.376291] Lustre: Skipped 2 previous similar messages [ 1151.378703] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1151.381632] Lustre: Skipped 3 previous similar messages [ 1151.384596] Lustre: 98430:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1153.296271] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1156.886356] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1157.636949] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1157.639469] Lustre: Skipped 2 previous similar messages [ 1157.641269] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1157.670260] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1158.257693] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1158.327187] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1159.704974] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1159.779366] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1169.913677] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1175.045866] Lustre: Failing over lustre-MDT0000 [ 1175.488043] Lustre: server umount lustre-MDT0000 complete [ 1176.569653] LustreError: 95009:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.203.38@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1178.082934] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1178.083247] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1178.087402] Lustre: Skipped 3 previous similar messages [ 1178.090787] LustreError: Skipped 1 previous similar message [ 1181.681328] LustreError: 95010:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.203.38@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1181.687048] LustreError: 95010:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 1190.239399] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1190.241100] LDISKFS-fs (dm-0): recovery complete [ 1190.243920] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1190.280720] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1190.403074] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1191.775695] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1191.920644] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1191.921830] Lustre: 96852:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1195.490161] Lustre: 96444:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1195.493819] Lustre: 96444:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 1 previous similar message [ 1195.500295] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1195.515531] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1195.832072] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1196.145978] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1196.149496] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 1196.464126] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1196.468439] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 1196.786136] Lustre: 101246:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1196.789843] Lustre: 101246:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 8 previous similar messages [ 1197.418777] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1197.423457] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 3 previous similar messages [ 1197.744099] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1197.746508] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 3 previous similar messages [ 1199.000765] Lustre: 101246:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1199.004275] Lustre: 101246:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 6 previous similar messages [ 1199.632779] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1199.638090] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 1199.952060] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1199.956333] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 1203.081949] Lustre: 101246:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1203.085854] Lustre: 101246:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 12 previous similar messages [ 1203.713966] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1203.717347] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 1204.032083] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1204.034618] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 1211.218044] Lustre: 101246:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1211.221627] Lustre: 101246:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 25 previous similar messages [ 1211.850278] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1211.853354] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 25 previous similar messages [ 1212.168139] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1212.172602] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 25 previous similar messages [ 1227.467227] Lustre: 101246:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1227.474071] Lustre: 101246:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 51 previous similar messages [ 1228.106108] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1228.109195] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 51 previous similar messages [ 1228.424066] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1228.428096] LustreError: 101246:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 51 previous similar messages [ 1251.208079] Lustre: 101246:0:(ldlm_lib.c:2394:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1251.211359] Lustre: 101246:0:(ldlm_lib.c:2404:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1251.214620] Lustre: 101246:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client cd4544e9-314b-4eb9-b5aa-e2cb7a8c3989@192.168.203.38@tcp [ 1251.218748] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1251.220996] Lustre: 101246:0:(ldlm_lib.c:1914:abort_req_replay_queue()) @@@ aborted: req@00000000ea9fb09a x1873325539263872/t0(8589934775) o101->cd4544e9-314b-4eb9-b5aa-e2cb7a8c3989@192.168.203.38@tcp:738/0 lens 592/0 e 0 to 0 dl 1786542893 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1251.230537] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1251.234340] Lustre: lustre-MDT0000: Denying connection for new client cd4544e9-314b-4eb9-b5aa-e2cb7a8c3989 (at 192.168.203.38@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1251.237495] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1251.253277] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1251.253445] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1251.256735] Lustre: Skipped 2 previous similar messages [ 1251.277621] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 1251.277729] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 1260.001590] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1260.006089] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1260.013164] Lustre: Skipped 2 previous similar messages [ 1260.016282] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1260.020808] Lustre: Skipped 4 previous similar messages [ 1265.533995] Lustre: server umount lustre-OST0000 complete [ 1267.174807] LustreError: 96834:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1267.179166] LustreError: 96834:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 11 previous similar messages [ 1280.992100] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1281.054442] Lustre: server umount lustre-OST0001 complete [ 1288.539567] Lustre: server umount lustre-MDT0000 complete [ 1289.761181] LustreError: 94995:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786542921 with bad export cookie 6309426792740439973 [ 1289.764442] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1289.765377] LustreError: 94995:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1289.885739] Lustre: server umount lustre-MDT0001 complete [ 1292.655896] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 09:55:24 (1786542924) [ 1295.191349] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1295.194538] Lustre: *** cfs_fail_loc=197, val=0*** [ 1310.624749] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1315.744388] LustreError: 103231:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.138@tcp: failed processing log, type 4: rc = -110 [ 1320.870766] Lustre: *** cfs_fail_loc=197, val=0*** [ 1320.871910] Lustre: Skipped 65 previous similar messages [ 1341.344408] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1341.346722] Lustre: Skipped 7 previous similar messages [ 1343.194600] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1345.570901] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1346.270459] Lustre: server umount lustre-OST0000 complete [ 1349.013886] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 09:56:20 (1786542980) [ 1362.598174] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1362.775531] LustreError: 104846:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1362.782733] LustreError: 104846:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 1364.105270] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1367.764825] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1369.754578] LustreError: 104879:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1369.759411] LustreError: 104879:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1369.762860] LustreError: 104879:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -5 [ 1369.896015] Lustre: server umount lustre-MDT0000 complete [ 1382.091690] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_hostid [ 1385.887494] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 1398.396841] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1398.504180] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1398.516807] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1398.558067] Lustre: lustre-MDT0000: new disk, initializing [ 1398.581606] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1400.070181] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1404.356663] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1404.415405] Lustre: 109044:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1404.537231] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1406.009789] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1409.745762] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1411.516894] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1413.783749] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1413.913802] Lustre: 111014:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1416.199937] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1420.066826] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1421.529025] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1422.872182] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1423.657925] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 1425.061835] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1425.138172] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1427.941274] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1427.941835] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1427.945120] LustreError: Skipped 4 previous similar messages [ 1427.945771] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1427.952446] Lustre: Skipped 6 previous similar messages [ 1427.957894] Lustre: Skipped 12 previous similar messages [ 1431.889236] Lustre: server umount lustre-OST0000 complete [ 1439.314791] Lustre: server umount lustre-MDT0000 complete [ 1440.616161] LustreError: 109038:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786543072 with bad export cookie 6309426792740450599 [ 1440.619969] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1440.622147] LustreError: 109038:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1440.757313] Lustre: server umount lustre-MDT0001 complete [ 1443.684292] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 09:57:55 (1786543075) [ 1446.689489] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 1447.767887] Key type lgssc unregistered [ 1447.887340] LNet: 113513:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1447.889874] LNetError: 113513:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1447.901276] LNet: Removed LNI 192.168.203.138@tcp [ 1448.215455] Key type .llcrypt unregistered [ 1448.216580] Key type ._llcrypt unregistered [ 1454.306102] Key type ._llcrypt registered [ 1454.308137] Key type .llcrypt registered [ 1454.351465] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_hostid [ 1460.466803] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 1460.812604] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1460.843162] alg: No test for adler32 (adler32-zlib) [ 1461.693561] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 1461.780358] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 1463.360156] Key type lgssc registered [ 1463.708375] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1476.330647] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1476.337228] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1477.429789] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1477.439417] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1477.477435] Lustre: lustre-MDT0000: new disk, initializing [ 1477.502383] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1477.511548] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1478.899625] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1482.943639] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1483.050094] Lustre: lustre-OST0000: new disk, initializing [ 1483.051787] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1483.053510] Lustre: Skipped 1 previous similar message [ 1483.055526] Lustre: 117657:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1483.080413] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1485.183898] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1486.853427] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1486.856710] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1486.865647] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1489.226471] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1489.289698] Lustre: lustre-OST0001: new disk, initializing [ 1489.292036] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1489.295867] Lustre: 118520:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1489.322741] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1491.159730] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1492.740891] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 1492.743884] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 1492.754716] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 1498.082244] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1498.083348] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1498.088785] Lustre: Skipped 1 previous similar message [ 1498.091779] Lustre: Skipped 1 previous similar message [ 1501.287684] Lustre: server umount lustre-MDT0000 complete [ 1503.743297] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1505.463891] LustreError: 116844:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786543137 with bad export cookie 7647852125829801429 [ 1505.467929] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1505.468402] LustreError: 116844:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1505.516502] Lustre: server umount lustre-OST0000 complete [ 1506.878988] Lustre: server umount lustre-OST0001 complete [ 1509.825643] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 09:59:01 (1786543141) [ 1517.452956] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 09:59:09 (1786543149) [ 1521.366107] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 1528.520607] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 09:59:20 (1786543160) [ 1540.670438] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_hostid [ 1543.817740] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 1555.199974] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1555.296188] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1555.305612] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1555.340722] Lustre: lustre-MDT0000: new disk, initializing [ 1555.360399] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1555.367868] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1556.707823] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1560.817482] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1560.856752] Lustre: 126254:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1560.878103] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1560.880023] Lustre: Skipped 1 previous similar message [ 1560.917340] Lustre: lustre-MDT0001: new disk, initializing [ 1560.949670] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1560.960884] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1560.964801] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1562.278968] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1566.087228] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1567.947422] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1570.347747] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1570.475473] Lustre: lustre-OST0000: new disk, initializing [ 1570.477242] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1570.480065] Lustre: 128226:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1570.514641] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1572.599147] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1575.938622] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1575.942199] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1575.984179] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1576.357620] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1577.939038] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1578.003327] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1579.439865] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1579.514533] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1581.028021] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1581.028154] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1581.033607] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1581.035013] LustreError: Skipped 1 previous similar message [ 1581.041241] Lustre: Skipped 1 previous similar message [ 1582.227678] Lustre: server umount lustre-OST0000 complete [ 1586.145703] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1586.146641] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1586.150489] Lustre: Skipped 2 previous similar messages [ 1586.154612] Lustre: Skipped 1 previous similar message [ 1589.652711] Lustre: server umount lustre-MDT0000 complete [ 1590.879265] LustreError: 126248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786543222 with bad export cookie 7647852125829802640 [ 1590.882796] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1590.883665] LustreError: 126248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1591.017239] Lustre: server umount lustre-MDT0001 complete [ 1593.852638] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 10:00:25 (1786543225) [ 1595.955738] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1596.139784] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1597.574258] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1600.349409] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1601.779977] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1604.132917] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1605.949432] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1608.141461] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1610.061623] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1612.453424] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1625.245076] Lustre: *** cfs_fail_loc=159, val=0*** [ 1641.471838] Lustre: lustre-MDT0000: Client dc85ed67-c304-42aa-9b7b-8d1afa54678c (at 192.168.203.38@tcp) reconnecting [ 1642.007306] Lustre: *** cfs_fail_loc=159, val=0*** [ 1642.008821] Lustre: Skipped 5 previous similar messages [ 1658.367640] Lustre: lustre-MDT0000: Client dc85ed67-c304-42aa-9b7b-8d1afa54678c (at 192.168.203.38@tcp) reconnecting [ 1661.409062] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1661.411555] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1661.415530] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1662.437676] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1665.748768] Lustre: server umount lustre-OST0000 complete [ 1667.041840] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1667.044224] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1667.048993] Lustre: Skipped 1 previous similar message [ 1667.050639] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1667.052544] Lustre: Skipped 2 previous similar messages [ 1677.793444] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1677.797359] Lustre: Skipped 4 previous similar messages [ 1681.376112] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1681.446952] Lustre: server umount lustre-MDT0000 complete [ 1682.707755] LustreError: 133688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786543314 with bad export cookie 7647852125829803508 [ 1682.711639] LustreError: 133688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1682.711665] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1682.837840] Lustre: server umount lustre-MDT0001 complete [ 1686.555077] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 1687.581701] Key type lgssc unregistered [ 1687.744444] LNet: 134443:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1687.749174] LNetError: 134443:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1687.757379] LNet: Removed LNI 192.168.203.138@tcp [ 1688.111222] Key type .llcrypt unregistered [ 1688.112880] Key type ._llcrypt unregistered [ 1697.008528] Key type ._llcrypt registered [ 1697.010461] Key type .llcrypt registered [ 1697.052556] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 1697.405313] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1697.438412] alg: No test for adler32 (adler32-zlib) [ 1698.299637] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 1698.390686] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 1699.968193] Key type lgssc registered [ 1700.340860] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1702.944784] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1702.950680] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1704.092495] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1705.460889] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1708.528353] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1708.670410] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1710.123113] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1712.960267] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1714.796793] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1717.144573] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1717.266055] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1719.180977] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1721.875781] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1730.602461] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 1737.223949] Lustre: *** cfs_fail_loc=159, val=0*** [ 1753.611147] Lustre: lustre-MDT0000: Client bc23aedc-9bb7-4cdc-b0c3-87d444288408 (at 192.168.203.38@tcp) reconnecting [ 1754.562865] Lustre: *** cfs_fail_loc=159, val=0*** [ 1754.565179] Lustre: Skipped 1 previous similar message [ 1769.984944] Lustre: lustre-MDT0001: Client bc23aedc-9bb7-4cdc-b0c3-87d444288408 (at 192.168.203.38@tcp) reconnecting [ 1770.503135] Lustre: *** cfs_fail_loc=159, val=0*** [ 1770.505379] Lustre: Skipped 1 previous similar message [ 1785.856333] Lustre: lustre-MDT0001: Client bc23aedc-9bb7-4cdc-b0c3-87d444288408 (at 192.168.203.38@tcp) reconnecting [ 1787.255758] Lustre: 138887:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1788.186718] Lustre: *** cfs_fail_loc=159, val=0*** [ 1788.190116] Lustre: Skipped 4 previous similar messages [ 1803.795437] Lustre: lustre-MDT0000: Client 894e61cd-a6d0-4884-98e9-9ca705fd5517 (at 192.168.203.38@tcp) reconnecting [ 1804.504502] Lustre: *** cfs_fail_loc=159, val=0*** [ 1804.506673] Lustre: Skipped 14 previous similar messages [ 1820.159075] Lustre: lustre-MDT0000: Client 894e61cd-a6d0-4884-98e9-9ca705fd5517 (at 192.168.203.38@tcp) reconnecting [ 1820.671650] Lustre: 139174:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1822.690745] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1822.691756] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1822.698784] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1822.700545] Lustre: Skipped 1 previous similar message [ 1827.811036] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1827.813044] Lustre: Skipped 1 previous similar message [ 1828.137384] Lustre: server umount lustre-OST0000 complete [ 1831.904980] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1831.907870] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1831.911932] Lustre: Skipped 1 previous similar message [ 1831.913672] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1832.931121] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1838.049798] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1838.052550] Lustre: Skipped 3 previous similar messages [ 1843.170360] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1843.173332] Lustre: Skipped 1 previous similar message [ 1844.192098] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1844.267522] Lustre: server umount lustre-MDT0000 complete [ 1845.540075] LustreError: 135813:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786543477 with bad export cookie 11755863780253829432 [ 1845.543806] LustreError: 135813:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1845.544181] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1845.665800] Lustre: server umount lustre-MDT0001 complete [ 1849.631239] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 1850.674576] Key type lgssc unregistered [ 1850.790350] LNet: 140228:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1850.793139] LNetError: 140228:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1850.802316] LNet: Removed LNI 192.168.203.138@tcp [ 1851.096888] Key type .llcrypt unregistered [ 1851.098944] Key type ._llcrypt unregistered [ 1860.833900] Key type ._llcrypt registered [ 1860.835240] Key type .llcrypt registered [ 1860.877256] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 1861.283570] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1861.292056] alg: No test for adler32 (adler32-zlib) [ 1862.140736] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 1862.232972] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 1863.816193] Key type lgssc registered [ 1864.224246] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1867.083891] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1867.093063] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1868.234956] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1869.852593] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1872.929229] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1873.058294] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1874.482036] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1877.376098] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1879.399539] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1882.106919] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1882.239803] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1883.314437] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 1884.513117] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1887.253314] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1893.555126] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 1900.452671] Lustre: 144222:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1901.110718] Lustre: 144271:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1901.574814] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 1902.854555] Lustre: 144420:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 1903.194682] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 1903.198052] Lustre: Skipped 1 previous similar message [ 1903.529377] Lustre: 144518:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1909.221798] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1909.222193] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1909.226050] LustreError: Skipped 1 previous similar message [ 1909.232969] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1909.237177] Lustre: Skipped 1 previous similar message [ 1910.731341] Lustre: server umount lustre-OST0000 complete [ 1914.338387] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1914.338797] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1914.350837] Lustre: Skipped 1 previous similar message [ 1914.354150] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1914.357377] Lustre: Skipped 1 previous similar message [ 1918.180223] Lustre: server umount lustre-MDT0000 complete [ 1919.395056] LustreError: 144234:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786543551 with bad export cookie 6294175857513275870 [ 1919.398190] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1919.398571] LustreError: 144234:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1919.512151] Lustre: server umount lustre-MDT0001 complete [ 1923.286765] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 1924.283641] Key type lgssc unregistered [ 1924.418407] LNet: 145577:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1924.420993] LNetError: 145577:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1924.430424] LNet: Removed LNI 192.168.203.138@tcp [ 1924.725600] Key type .llcrypt unregistered [ 1924.726739] Key type ._llcrypt unregistered [ 1934.832957] Key type ._llcrypt registered [ 1934.834166] Key type .llcrypt registered [ 1934.868179] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 1935.218487] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1935.249241] alg: No test for adler32 (adler32-zlib) [ 1936.097520] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 1936.182483] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 1937.768228] Key type lgssc registered [ 1938.184654] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1940.749424] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1940.758935] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1941.904512] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1943.276488] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1946.342260] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1946.500430] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1947.931841] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1950.646703] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1953.028342] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1956.032157] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1956.215135] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1957.231474] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 1958.919470] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 1962.022932] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1966.446493] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 1969.865738] Lustre: *** cfs_fail_loc=159, val=0*** [ 1986.053115] Lustre: lustre-MDT0000: Client 1871f4a2-f727-47dd-ad05-22cd6980b051 (at 192.168.203.38@tcp) reconnecting [ 1988.065121] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1988.067927] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1988.072546] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1992.166714] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1992.167774] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1992.167838] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1993.623440] Lustre: server umount lustre-OST0000 complete [ 1997.283083] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1997.283925] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1997.287884] Lustre: Skipped 1 previous similar message [ 1997.291799] Lustre: Skipped 2 previous similar messages [ 2001.325451] Lustre: server umount lustre-MDT0000 complete [ 2002.400964] LustreError: 146968:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2002.405940] LustreError: 146968:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2002.667402] LustreError: 146949:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786543634 with bad export cookie 16425212522732138095 [ 2002.672711] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2002.672842] LustreError: 146949:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2002.798337] Lustre: server umount lustre-MDT0001 complete [ 2006.747989] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 2007.828158] Key type lgssc unregistered [ 2007.954450] LNet: 150633:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2007.957384] LNetError: 150633:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2007.967302] LNet: Removed LNI 192.168.203.138@tcp [ 2008.278888] Key type .llcrypt unregistered [ 2008.280208] Key type ._llcrypt unregistered [ 2018.166198] Key type ._llcrypt registered [ 2018.167232] Key type .llcrypt registered [ 2018.203345] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 2018.518402] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2018.550760] alg: No test for adler32 (adler32-zlib) [ 2019.418084] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 2019.500866] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 2021.080227] Key type lgssc registered [ 2021.424606] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2024.706108] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2024.721530] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2025.886130] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2027.272873] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 2030.764909] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2030.941234] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2032.606808] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 2035.339518] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2037.267209] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2039.517018] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2039.643422] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2040.678656] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2041.649109] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 2044.020469] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2050.920955] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2067.564314] Lustre: lustre-OST0000: evicting ac7a8dd5-3e63-4bc1-a247-20a47e2f8d65 (at 192.168.203.38@tcp) by administrative request [ 2067.568379] Lustre: Skipped 1 previous similar message [ 2092.002486] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2092.003240] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2092.004420] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2092.008250] Lustre: Skipped 1 previous similar message [ 2092.011836] LustreError: Skipped 1 previous similar message [ 2097.122128] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2097.124153] Lustre: Skipped 2 previous similar messages [ 2097.747265] Lustre: server umount lustre-OST0000 complete [ 2102.241595] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2102.242486] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2102.246449] Lustre: Skipped 1 previous similar message [ 2105.305227] Lustre: server umount lustre-MDT0000 complete [ 2106.536179] LustreError: 152006:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786543738 with bad export cookie 12700778496162651599 [ 2106.538380] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2106.539691] LustreError: 152006:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2106.667610] Lustre: server umount lustre-MDT0001 complete [ 2110.344072] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 2111.339650] Key type lgssc unregistered [ 2111.450392] LNet: 156114:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2111.453228] LNetError: 156114:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2111.461302] LNet: Removed LNI 192.168.203.138@tcp [ 2111.752481] Key type .llcrypt unregistered [ 2111.753683] Key type ._llcrypt unregistered [ 2135.982257] Key type ._llcrypt registered [ 2135.983502] Key type .llcrypt registered [ 2136.021114] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 2136.432859] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2136.438408] alg: No test for adler32 (adler32-zlib) [ 2137.281098] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 2137.362874] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 2138.944153] Key type lgssc registered [ 2139.269880] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2142.067753] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2142.077719] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2143.235912] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2144.883938] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 2148.414986] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2148.565533] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2150.003821] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 2152.436972] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2154.325876] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2156.932065] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2157.059713] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2158.121982] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 2159.237243] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 2161.800354] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2168.358332] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2178.530196] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2178.531303] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2178.533304] LustreError: Skipped 1 previous similar message [ 2178.537577] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2178.542344] Lustre: Skipped 1 previous similar message [ 2181.170238] Lustre: server umount lustre-OST0000 complete [ 2183.652709] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2183.653925] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2183.661147] Lustre: Skipped 2 previous similar messages [ 2188.770677] LustreError: 158572:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2188.776442] LustreError: 158572:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2188.787916] Lustre: server umount lustre-MDT0000 complete [ 2190.051299] LustreError: 161082:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786543822 with bad export cookie 7999994208503217041 [ 2190.055566] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2190.056966] LustreError: 161082:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2190.188411] Lustre: server umount lustre-MDT0001 complete [ 2193.754488] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 2194.901743] Key type lgssc unregistered [ 2195.033519] LNet: 162135:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2195.036044] LNetError: 162135:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2195.043397] LNet: Removed LNI 192.168.203.138@tcp [ 2195.382101] Key type .llcrypt unregistered [ 2195.383885] Key type ._llcrypt unregistered [ 2204.663403] Key type ._llcrypt registered [ 2204.664457] Key type .llcrypt registered [ 2204.706345] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 2205.062087] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2205.118521] alg: No test for adler32 (adler32-zlib) [ 2205.987057] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 2206.081927] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 2207.664208] Key type lgssc registered [ 2208.049122] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2212.941510] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 10:10:44 (1786543844) [ 2218.392418] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_hostid [ 2221.726315] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 2234.405435] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2234.414406] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2235.499468] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2235.510285] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2235.548883] Lustre: lustre-MDT0000: new disk, initializing [ 2235.568798] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2235.574547] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2236.876765] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 2241.025835] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2241.066984] Lustre: 166370:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2241.088270] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2241.090123] Lustre: Skipped 1 previous similar message [ 2241.124694] Lustre: lustre-MDT0001: new disk, initializing [ 2241.155977] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2241.166445] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2241.169906] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2242.540098] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 2246.470173] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2248.508463] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2251.033490] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2251.174234] Lustre: lustre-OST0000: new disk, initializing [ 2251.177233] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2251.181649] Lustre: 168340:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2251.230265] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2253.225194] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 2256.921709] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2257.399561] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2257.402608] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2257.436809] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2267.618148] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2267.618622] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2267.620493] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2267.620838] LustreError: Skipped 1 previous similar message [ 2267.629631] Lustre: Skipped 1 previous similar message [ 2272.018537] Lustre: server umount lustre-OST0000 complete [ 2277.344942] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2277.347575] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2277.353568] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2277.355732] Lustre: Skipped 1 previous similar message [ 2279.392189] Lustre: server umount lustre-MDT0000 complete [ 2280.707549] LustreError: 166363:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786543912 with bad export cookie 181685632098392565 [ 2280.712379] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2280.714013] LustreError: 166363:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2280.837419] Lustre: server umount lustre-MDT0001 complete [ 2284.662183] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 2285.875194] Key type lgssc unregistered [ 2286.043427] LNet: 170247:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2286.046892] LNetError: 170247:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2286.055426] LNet: Removed LNI 192.168.203.138@tcp [ 2286.404706] Key type .llcrypt unregistered [ 2286.405800] Key type ._llcrypt unregistered [ 2295.637451] Key type ._llcrypt registered [ 2295.638587] Key type .llcrypt registered [ 2295.674443] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 2296.096195] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2296.102978] alg: No test for adler32 (adler32-zlib) [ 2296.953354] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 2297.040867] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 2298.624192] Key type lgssc registered [ 2299.006360] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2301.778719] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2301.786174] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2302.930826] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2304.219394] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 2307.400328] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2307.547163] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2309.009041] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 2311.960344] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2314.235559] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2317.581587] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2317.745259] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2320.081843] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 2322.647213] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2327.977797] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 2330.384993] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2330.397541] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2330.408081] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2330.412802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2330.418759] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2330.788831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2331.099340] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2331.101596] Lustre: Skipped 1 previous similar message [ 2331.107481] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2331.112744] Lustre: Skipped 1 previous similar message [ 2331.115075] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2331.118361] Lustre: Skipped 1 previous similar message [ 2331.120446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2331.123939] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2331.126111] Lustre: Skipped 1 previous similar message [ 2331.452543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2331.829277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2332.182410] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2332.187574] Lustre: Skipped 2 previous similar messages [ 2332.195790] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2332.201177] Lustre: Skipped 2 previous similar messages [ 2332.203276] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2332.206287] Lustre: Skipped 2 previous similar messages [ 2332.208347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2332.211696] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2332.213488] Lustre: Skipped 2 previous similar messages [ 2332.574647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2332.961355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2333.311644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2333.648088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2334.019363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2334.339682] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2334.343853] Lustre: Skipped 5 previous similar messages [ 2334.350411] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2334.354533] Lustre: Skipped 5 previous similar messages [ 2334.356334] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2334.358819] Lustre: Skipped 5 previous similar messages [ 2334.360688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2334.364406] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2334.366616] Lustre: Skipped 5 previous similar messages [ 2334.677289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2335.031600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2335.337195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2335.671316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2335.971400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2336.271055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2336.593404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2336.955988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2337.360886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2337.728822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2338.166494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2338.533971] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2338.537735] Lustre: Skipped 11 previous similar messages [ 2338.545739] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2338.552938] Lustre: Skipped 11 previous similar messages [ 2338.555830] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2338.560032] Lustre: Skipped 11 previous similar messages [ 2338.562692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2338.568985] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2338.572523] Lustre: Skipped 11 previous similar messages [ 2339.008552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2339.451524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2339.819144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2340.206763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2340.616844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2340.982127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2341.387657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2341.742173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2342.080861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2342.464490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2342.837428] LustreError: 171656:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2342.842633] LustreError: 171656:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2342.842833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2343.230134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2343.633668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2344.034428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2344.384864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2344.718673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2345.117954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2345.452650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2345.852749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2346.241505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2346.556613] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2346.559581] Lustre: Skipped 20 previous similar messages [ 2346.565872] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2346.570052] Lustre: Skipped 20 previous similar messages [ 2346.571685] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2346.573909] Lustre: Skipped 20 previous similar messages [ 2346.575409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2346.579065] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2346.581386] Lustre: Skipped 20 previous similar messages [ 2346.893957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2347.189721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2347.515966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2347.864972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2348.226765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2348.650435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2348.998934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2349.337439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2349.689238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2350.009313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2350.365635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2350.780444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2351.177528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2351.562614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2351.959760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2352.308421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2352.659571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2353.023239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2353.427312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2353.878978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2354.267286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2354.650399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2355.005973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2355.402150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2355.722841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2356.054053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2356.387845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2356.718151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2357.035740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2357.333427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2357.653223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2357.955752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2358.258927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2358.625190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2359.036430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2359.405348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2359.719270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2360.014651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2360.337331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2360.635729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2360.911402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2361.189258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2361.485027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2361.778888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2362.093249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2362.415202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2362.767759] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2362.770396] Lustre: Skipped 46 previous similar messages [ 2362.775893] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2362.780619] Lustre: Skipped 46 previous similar messages [ 2362.782378] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2362.785215] Lustre: Skipped 46 previous similar messages [ 2362.787217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2362.791334] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2362.793336] Lustre: Skipped 46 previous similar messages [ 2363.117238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2363.407110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2363.716207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2364.076781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2364.427387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2364.759252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2365.060200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2365.356660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2365.672102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2366.007832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2366.325905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2366.671624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2366.965319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2367.268732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2367.559058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2367.844253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2368.133193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2368.427647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2368.722074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2369.050303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2369.323981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2369.601296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2369.898790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2370.196560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2370.491667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2370.814965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2371.108381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2371.424183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2371.724285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2372.010472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2372.314368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2372.613645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2372.905095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2373.217843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2373.538163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2373.844185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2374.123963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2374.424366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2374.748243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2375.095227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2375.415633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2375.739470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2376.047493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2376.359625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2376.672031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2376.988485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2377.283771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2377.626157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2377.979424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2378.339668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2378.725599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2379.070756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2379.369536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2379.720582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2380.074410] LustreError: 171656:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -108 [ 2380.082319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2380.082782] LustreError: 171656:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -108 [ 2380.460768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2380.843185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2381.277493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2381.660556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2382.036777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2382.454667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2382.752132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2383.086001] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2383.466477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2383.822249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2384.118421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2384.459562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2384.859465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2385.280107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2385.694757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2386.149482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2386.566613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2386.984838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2387.385193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2387.848351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2388.235412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2388.640298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2388.965057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2389.304059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2389.713735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2390.018242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2390.322575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2390.624074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2390.929402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2391.238267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2391.591148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2392.021434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2392.469739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2392.917739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2393.279660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2393.646229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2393.983148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2394.375086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2394.757590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2395.093639] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2395.097433] Lustre: Skipped 94 previous similar messages [ 2395.102627] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2395.106303] Lustre: Skipped 94 previous similar messages [ 2395.108031] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2395.110062] Lustre: Skipped 94 previous similar messages [ 2395.111466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2395.116375] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2395.119842] Lustre: Skipped 94 previous similar messages [ 2395.406813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2395.804854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2396.154176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2396.492792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2396.795832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2397.100181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2397.395857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2397.694803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2398.005309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2398.317845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2398.603224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2398.898213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2399.188344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2399.494746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2399.790097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2400.076892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2400.372588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2400.669464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2400.951759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2401.240637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2401.540767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2401.863433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2402.176193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2402.552313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2402.941223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2403.258448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2403.641533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2404.020037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2404.380978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2404.700826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2405.014212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2405.382245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2405.729568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2406.059688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2406.367506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2406.680611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2406.991982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2407.316067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2407.643149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2407.950330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2408.248180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2408.543634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2408.836478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2409.149841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2409.434578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2409.716935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2410.002636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2410.303469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2410.589650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2410.879790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2411.181969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2411.498504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2411.801696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2412.098182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2412.412413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2412.724765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2413.053723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2413.362921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2413.675848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2414.007576] LustreError: 171656:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2414.011712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2414.012388] LustreError: 171656:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2414.333237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2414.666198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2414.983996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2415.329694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2415.656061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2416.002162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2416.351328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2416.735098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2417.082471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2417.544434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2418.059892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2418.575369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2419.120072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2419.752666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2420.309851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2420.885167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2421.463084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2422.066706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2422.649300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2423.075408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2423.511309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2423.983643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2424.392421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2424.890231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2425.325169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2425.758342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2426.195187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2426.686254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2427.123775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2427.499814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2427.911711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2428.291174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2428.672287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2429.138996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2429.591763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2430.044640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2430.548810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2431.097044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2431.653649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2432.220279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2432.695155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2433.238884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2433.757629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2434.246060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2434.691408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2435.121346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2435.596147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2436.004859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2436.395938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2436.760652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2437.139112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2437.563437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2437.949591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2438.282213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2438.581801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2438.878415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2439.176393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2439.471222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2439.762569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2440.060226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2440.345045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2440.645652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2440.936283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2441.224082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2441.497706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2441.773612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2442.044384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2442.312884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2442.585954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2442.873941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2443.145546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2443.430668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2443.705867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2443.995866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2444.288123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2444.576570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2444.876624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2445.191333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2445.473758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2445.751345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2446.021195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2446.294889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2446.566506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2446.839567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2447.108334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2447.375640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2447.642058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2447.929532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2448.243152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2448.550240] LustreError: 171656:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2448.553884] LustreError: 171656:0:(osp_precreate.c:655:osp_precreate_send()) Skipped 1 previous similar message [ 2448.554242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2448.556563] LustreError: 171656:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2448.563530] LustreError: 171656:0:(osp_precreate.c:1359:osp_precreate_thread()) Skipped 1 previous similar message [ 2448.852994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2449.169563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2449.479151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2449.801074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2450.086607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2450.390556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2450.683127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2450.997187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2451.293713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2451.586314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2451.881762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2452.183240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2452.478335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2452.805056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2453.105261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2453.407252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2453.707511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2454.006621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2454.308215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2454.635073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2454.942187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2455.235661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2455.545181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2455.887096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2456.196650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2456.518203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2456.826142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2457.137021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2457.432301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2457.716077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2458.003848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2458.308874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2458.620497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2458.924260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2459.235499] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2459.237600] Lustre: Skipped 184 previous similar messages [ 2459.242356] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2459.247364] Lustre: Skipped 184 previous similar messages [ 2459.249549] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2459.251620] Lustre: Skipped 184 previous similar messages [ 2459.252939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2459.256853] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2459.259043] Lustre: Skipped 184 previous similar messages [ 2459.574623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2459.883898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2460.199071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2460.515894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2460.836652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2461.137603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2461.445734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2461.760628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2462.089233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2462.427678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2462.711328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2462.984134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2463.262629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2463.563861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2463.850167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2464.113330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2464.383563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2464.669208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2464.964649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2465.236281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2465.529084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2465.812069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2466.098413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2466.390493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2466.691886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2466.961507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2467.236268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2467.512889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2467.798503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2468.092906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2468.383949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2468.685986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2468.990198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2469.274646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2469.567505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2469.848198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2470.145216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2470.422537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2470.733541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2471.026163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2471.329317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2471.621257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2471.917701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2472.221323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2472.534601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2472.847422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2473.127449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2473.406636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2473.727448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.035537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.329652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.617474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.907184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.216870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.505773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.784560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.057550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.341376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.631395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.909843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.185572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.457080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.727380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.006384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.287409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.574571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.844525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.106854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.377021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.658567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.931166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.216462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.499357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.786170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.061350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.330661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.610642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.888711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2482.172107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2482.455181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2482.746723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.053880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.318657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.600866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.883217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.172891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.473310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.765952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.064922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.351302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.618168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.891045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.165096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.433115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.702845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.991396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.256953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.532805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.819958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.097198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.380100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.653100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.932081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.207666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.485300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.764017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.035594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.312845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.590320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.871178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2491.146141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2491.414119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2491.689476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2491.963494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.222418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.496190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.760454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.057416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.341491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.607987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.889385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.182820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.465590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.739035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.014651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.299546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.570259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.842996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.120696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.399706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.679645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.968931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.261209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.546201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.814457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.101153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.375522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.649972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.938817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.230410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.526284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.834878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.160172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.476380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.779978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.087418] LustreError: 171656:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2501.091777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.092152] LustreError: 171656:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2501.398900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.692608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.010754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.336327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.640522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.928704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2503.220225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2503.544293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2503.857817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.172977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.456874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.755824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.074986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.388639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.697220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.019820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.323497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.643385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.972247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2507.278781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2507.555970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2507.840738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.125716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.420050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.697232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.976397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.253799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.537866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.827826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.132578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.433213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.738520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.057076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.368872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.662887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.965549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.247488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.529463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.808723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.119446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.400582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.691193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.972467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.246725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.519439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.790016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.062698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.352374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.630073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.898968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2516.163405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2516.446177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2516.725990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.004219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.270209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.549116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.837290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.127930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.404958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.690918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.968256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.276126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.593385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.917070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.217469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.528408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.844645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2521.157162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2521.442587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2521.729256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.047206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.373922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.669416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.977969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.286975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.607980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.921215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2524.220904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2524.503519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2524.834965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.158991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.477540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.808217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.139659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.435810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.713354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.992310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.294018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.588538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.871883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.158422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.445250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.738617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.021725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.314188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.608384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.877143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.157470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.435147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.720745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.993114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2531.272488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2531.550585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2531.828044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.101401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.402597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.703737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.037197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.305294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.569137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.848089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.120944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.393194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.690421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.983813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.290922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.581187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.878040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2536.169640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2536.464291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2536.758689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.065367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.365337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.678868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.967884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.265665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.555717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.858292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.150838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.447898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.764569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.089287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.408043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.731733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.047737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.390187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.695675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.007466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.320753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.659754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.976663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.288529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.611619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.937432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.247651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.545711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.850819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.193769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.500175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.813464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.137038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.466786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.802255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.112407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.428045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.745023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.046335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.353149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.660091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.991486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.289955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.608995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.912570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.231713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.538851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.847253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.134825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.438439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.733600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.028249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.337027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.650744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.953313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.269605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.596130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.956423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.258466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.557678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.875520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.209877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.510334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.837135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.150837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.460941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.768261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.081129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.394544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.721936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.022557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.321949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.622037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.943553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.252238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.549205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.863859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.197809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.529648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.846172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.153459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.483440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.781285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.081651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.392048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.721870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.021975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.323553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.629935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.948311] LustreError: 171656:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2563.951914] LustreError: 171656:0:(osp_precreate.c:655:osp_precreate_send()) Skipped 1 previous similar message [ 2563.952408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.955261] LustreError: 171656:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2563.964827] LustreError: 171656:0:(osp_precreate.c:1359:osp_precreate_thread()) Skipped 1 previous similar message [ 2564.256738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.570049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.885105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.228746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.546851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.868681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.172548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.490081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.778595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.069894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.353149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.665207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.959093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.255358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.561397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.876263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.172915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.472248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.775054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.074775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.366961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.658440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.973496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.265252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.566350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.857334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.146978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.442400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.731041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.020990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.315947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.610701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.900443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.191167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.494516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.804590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.107895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.408624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.703896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.017537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.307399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.604759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.901611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.204621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.507931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.839659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.133921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.433576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.768937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.113722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.404248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.714901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.006320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.305344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.602644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.907226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.193120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.472745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.766566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.085239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.373623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.672599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.970764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.275021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.565662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.865480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.156306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.463837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.758870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.054578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.351239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.651593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.946074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.246942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.546263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.844365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.133688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.420217] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2587.422291] Lustre: Skipped 431 previous similar messages [ 2587.428416] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2587.432377] Lustre: Skipped 431 previous similar messages [ 2587.433969] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2587.436822] Lustre: Skipped 431 previous similar messages [ 2587.438922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.443044] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2587.446020] Lustre: Skipped 431 previous similar messages [ 2587.730227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.040031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.328100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.653317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.969267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.274195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.579530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.874587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.172237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.477995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.766140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.062868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.360420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.658131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.948747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.233782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.529473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.832029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.121197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.419795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.714741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.014078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.301690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.590038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.882717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.182852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.484416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.779134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.063614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.369035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.655668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.953380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.259993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.557723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.851921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.156428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.469128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.776163] LustreError: 171656:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2598.779348] LustreError: 171656:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2598.780537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.084245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.389876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.695720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.011563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.327374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.632322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.926651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.236263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.536307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.837809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.134995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.435109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.724819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.019398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.307472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.613311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.921676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.211476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.517517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.826509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.118307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.400209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.690155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.989084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.277172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.576364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.875368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.175593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.467950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.758413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.068243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.387415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.692724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.987947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.278797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.584492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.881937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.173568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.474013] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.780501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.064672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.365415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.661530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.964956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.257092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.552928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.859610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.192370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.514703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.824972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.118219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.421320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.713549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.001625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.299327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.604817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.898214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.204147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.515300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.845143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.160050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.478968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.806658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.114302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.419656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.719512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.027407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.335600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.634608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.921088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.218385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.531364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.818346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.116940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.421313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.735361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.028987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.316364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.616757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.935276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.239621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.569499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.896787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.207492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.499618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.803508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.106889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.430956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.766336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.081442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.400616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.746046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.083125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.398405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.699957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.019500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.310754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.590140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.911721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.222771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.515576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.811961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.108877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.418624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.709978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.009601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.301536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.603516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.930911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.246243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.531094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.832832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.133928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.439789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.750194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.081753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.400398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.718370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.056476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.383516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.675323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.971289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.273762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.580319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.881450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.190594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.503516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.833501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.153200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.458090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.753847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.069293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.365721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.671439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.972905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.288547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.587906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.897344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.211197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.517582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.817024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.108037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.406754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.708681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.007663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.317058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.613733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.927641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.227766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.523561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.816232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.123994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.427946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.723160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.014948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.320281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.600536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.903944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.614654] LustreError: 174124:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 2712.619455] LustreError: 171656:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 2782.691484] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2782.692597] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2782.697885] Lustre: Skipped 1 previous similar message [ 2786.299880] Lustre: server umount lustre-OST0000 complete [ 2787.810467] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2787.812723] Lustre: Skipped 1 previous similar message [ 2792.929393] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2792.932703] Lustre: Skipped 1 previous similar message [ 2793.745922] Lustre: server umount lustre-MDT0000 complete [ 2795.057686] LustreError: 171618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786544427 with bad export cookie 18085897921476638083 [ 2795.061510] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2795.062282] LustreError: 171618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2795.181988] Lustre: server umount lustre-MDT0001 complete [ 2798.993738] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 2800.236422] Key type lgssc unregistered [ 2800.370321] LNet: 226643:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2800.373232] LNetError: 226643:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2800.380422] LNet: Removed LNI 192.168.203.138@tcp [ 2800.794777] Key type .llcrypt unregistered [ 2800.796720] Key type ._llcrypt unregistered [ 2810.050186] Key type ._llcrypt registered [ 2810.051361] Key type .llcrypt registered [ 2810.083746] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 2810.475297] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2810.487790] alg: No test for adler32 (adler32-zlib) [ 2811.352327] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 2811.451435] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 2813.032141] Key type lgssc registered [ 2813.388935] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2816.133379] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2816.139477] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2817.294548] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2818.641386] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 2821.434433] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2821.577398] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2822.980595] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 2825.481405] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2827.245332] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2829.596019] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2829.704590] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2831.651170] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 2834.235529] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2843.050891] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 2846.546595] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2847.112287] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2847.117283] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2847.120287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.125306] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2847.125332] LustreError: 228053:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 2848.227860] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2848.229406] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2848.232422] Lustre: Skipped 1 previous similar message [ 2848.236366] Lustre: Skipped 1 previous similar message [ 2849.788563] LustreError: 230528:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 192.168.203.38@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2849.798336] LustreError: 230528:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 2849.826854] Lustre: server umount lustre-OST0000 complete [ 2867.200193] LustreError: 228053:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 2878.977492] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2879.094345] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2881.308817] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 2884.303794] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2888.293687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2888.298647] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2888.298671] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2891.034347] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 6 sec [ 2893.160869] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff954bc3d67800.ost_server_uuid 50 [ 2893.641690] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff954bc3d67800.ost_server_uuid in FULL state after 0 sec [ 2898.917535] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2898.917535] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2898.917845] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2898.919050] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2900.768450] Lustre: server umount lustre-OST0000 complete [ 2903.521491] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2903.527127] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2903.537446] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2903.542151] Lustre: Skipped 1 previous similar message [ 2908.398660] Lustre: server umount lustre-MDT0000 complete [ 2909.154456] LustreError: 228034:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2909.165596] LustreError: 228034:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2910.124597] LustreError: 228016:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786544542 with bad export cookie 859525802972557646 [ 2910.131555] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2910.132923] LustreError: 228016:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2910.271508] Lustre: server umount lustre-MDT0001 complete [ 2914.556353] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 2915.631438] Key type lgssc unregistered [ 2915.774460] LNet: 233265:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2915.777956] LNetError: 233265:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2915.787590] LNet: Removed LNI 192.168.203.138@tcp [ 2916.226328] Key type .llcrypt unregistered [ 2916.228450] Key type ._llcrypt unregistered [ 2929.976628] Key type ._llcrypt registered [ 2929.977852] Key type .llcrypt registered [ 2930.018755] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 2930.345773] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2930.380449] alg: No test for adler32 (adler32-zlib) [ 2931.234640] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 2931.338690] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 2932.920250] Key type lgssc registered [ 2933.480573] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2937.273326] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2937.283782] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2938.458728] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2940.190779] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 2944.310054] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2944.463473] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2946.076908] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 2947.095671] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 2949.657121] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2949.826442] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2952.344504] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 2955.356670] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2955.430620] Lustre: lustre-OST0001: new disk, initializing [ 2955.432699] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2955.435775] Lustre: 237278:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2955.473940] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2957.766077] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 2958.304398] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2958.308217] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2958.344250] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2963.441475] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 2963.811827] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 2970.718807] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 2978.790517] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2978.794831] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2978.799710] Lustre: Skipped 3 previous similar messages [ 2978.804187] Lustre: Skipped 2 previous similar messages [ 2983.909472] LustreError: 234989:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2983.917523] LustreError: 234989:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2983.942097] Lustre: server umount lustre-MDT0000 complete [ 2985.666173] LustreError: 234953:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786544617 with bad export cookie 14898858763181609862 [ 2985.669490] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2985.671965] LustreError: 234953:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2985.784592] Lustre: server umount lustre-MDT0001 complete [ 2997.259764] Lustre: server umount lustre-OST0000 complete [ 3008.201433] Lustre: server umount lustre-OST0001 complete [ 3017.165070] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 3020.860323] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3021.050502] LustreError: 240387:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3021.069536] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3022.623933] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3026.402819] LustreError: 240388:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3026.436937] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3028.049269] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3029.105479] Lustre: 241529:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3031.593368] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3031.715864] LustreError: 241884:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3031.743464] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3031.747204] Lustre: Skipped 1 previous similar message [ 3033.939110] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3036.902879] LustreError: 241885:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3036.906834] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 3036.909971] LustreError: 241885:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3037.222949] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3039.163246] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3042.283372] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3042.795114] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 3043.384748] Lustre: 243399:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3047.393041] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3047.395981] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3047.400540] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3052.586626] Lustre: server umount mylustre-MDT0000 complete [ 3053.025317] LustreError: 240388:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3053.030981] LustreError: 240388:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3053.904430] LustreError: 240362:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786544685 with bad export cookie 14898858763181612655 [ 3053.907969] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3053.908385] LustreError: 240362:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3054.024261] Lustre: server umount mylustre-MDT0001 complete [ 3065.346813] Lustre: server umount mylustre-OST0000 complete [ 3077.116802] Lustre: server umount mylustre-OST0001 complete [ 3085.148862] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 3089.444693] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3089.618126] LustreError: 245703:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3089.637200] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3089.639927] Lustre: Skipped 1 previous similar message [ 3091.346553] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3095.092383] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3097.065106] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3098.196772] Lustre: 246844:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3100.672421] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3102.986787] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3106.810348] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3109.538707] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3114.491067] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 3114.491153] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 3118.700881] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3120.186538] Lustre: 248715:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3124.708271] Lustre: tfs-MDT0000-lwp-OST0000: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3124.710908] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 3124.713331] Lustre: Skipped 5 previous similar messages [ 3124.716159] Lustre: Skipped 4 previous similar messages [ 3128.884474] Lustre: server umount tfs-MDT0000 complete [ 3129.826998] LustreError: 247005:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3129.837851] LustreError: 247005:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 3130.767648] LustreError: 245678:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786544762 with bad export cookie 14898858763181614545 [ 3130.776123] LustreError: 245678:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3130.776128] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3130.967190] Lustre: server umount tfs-MDT0001 complete [ 3142.167676] Lustre: server umount tfs-OST0000 complete [ 3154.357846] Lustre: server umount tfs-OST0001 complete [ 3162.978715] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 3167.095818] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3167.291341] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3167.293841] Lustre: Skipped 3 previous similar messages [ 3168.831354] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3172.548639] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3174.519806] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3175.857217] Lustre: 252107:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3178.291624] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3180.497014] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3183.570703] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3185.512891] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3192.813416] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 3192.816105] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 3200.168758] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3201.670844] Lustre: 253997:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3204.360310] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 10:27:16 (1786544836) [ 3218.405295] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3218.409287] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3218.415035] Lustre: Skipped 3 previous similar messages [ 3218.418985] Lustre: Skipped 3 previous similar messages [ 3222.461897] Lustre: server umount lustre-MDT0000 complete [ 3223.522839] LustreError: 250965:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3223.535877] LustreError: 250965:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 3224.219138] LustreError: 251709:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786544856 with bad export cookie 14898858763181616211 [ 3224.224472] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3224.227178] LustreError: 251709:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 3228.643389] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3228.645422] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3228.652169] Lustre: Skipped 2 previous similar messages [ 3228.656872] Lustre: Skipped 2 previous similar messages [ 3230.429565] Lustre: server umount lustre-MDT0001 complete [ 3238.233345] Lustre: server umount lustre-OST0000 complete [ 3245.679411] Lustre: server umount lustre-OST0001 complete [ 3247.848443] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_hostid [ 3251.020193] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 3265.685378] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 3265.685378] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3265.685378] [ 3265.695362] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3265.795719] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3265.812647] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3265.862145] Lustre: lustre-MDT0000: new disk, initializing [ 3265.884883] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3265.887534] Lustre: Skipped 3 previous similar messages [ 3265.894087] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3267.428446] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3271.981118] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 3271.981118] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3271.981118] [ 3271.989594] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3272.036988] Lustre: 257173:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3272.064251] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3272.066267] Lustre: Skipped 1 previous similar message [ 3272.107505] Lustre: lustre-MDT0001: new disk, initializing [ 3272.153112] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3272.156472] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3273.637253] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3278.498247] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3278.665820] Lustre: lustre-OST0000: new disk, initializing [ 3278.669211] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3278.674197] Lustre: 258759:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3280.321516] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3280.327208] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3280.368269] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3281.093534] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3285.865886] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3285.957455] Lustre: lustre-OST0001: new disk, initializing [ 3285.960429] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3285.964689] Lustre: 259629:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3287.813117] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3287.816796] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3287.849578] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3288.067937] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3292.265674] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 10:28:44 (1786544924) [ 3296.324399] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 10:28:48 (1786544928) [ 3308.518579] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3308.520653] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3308.524376] Lustre: Skipped 1 previous similar message [ 3308.528179] Lustre: Skipped 2 previous similar messages [ 3314.241989] Lustre: server umount lustre-OST0000 complete [ 3318.241414] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3322.164816] Lustre: server umount lustre-MDT0000 complete [ 3323.769320] LustreError: 257167:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786544955 with bad export cookie 14898858763181617765 [ 3323.772984] LustreError: 257167:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3323.773173] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3328.993945] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3328.997292] Lustre: Skipped 7 previous similar messages [ 3338.208248] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3338.328922] Lustre: server umount lustre-MDT0001 complete [ 3342.686755] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 3414.175848] Lustre: server umount lustre-OST0001 complete [ 3416.683980] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_hostid [ 3423.792774] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 3438.544572] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3438.668945] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3438.684341] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3438.736618] Lustre: lustre-MDT0000: new disk, initializing [ 3438.771613] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3438.776066] Lustre: Skipped 3 previous similar messages [ 3438.786704] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3440.539210] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3446.128854] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3446.189196] Lustre: 264402:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3446.344235] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3448.138388] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3453.088646] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3455.520624] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3458.481950] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3458.597769] Lustre: lustre-OST0000: new disk, initializing [ 3458.599332] Lustre: Skipped 1 previous similar message [ 3458.601284] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3458.603670] Lustre: Skipped 2 previous similar messages [ 3458.607460] Lustre: 266373:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3459.733291] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3459.739192] Lustre: Skipped 1 previous similar message [ 3459.742718] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3459.785384] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3461.040051] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3465.788709] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3470.307532] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3470.308706] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3470.309410] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3470.309416] Lustre: Skipped 2 previous similar messages [ 3470.311323] LustreError: Skipped 1 previous similar message [ 3470.320566] Lustre: Skipped 5 previous similar messages [ 3474.672016] Lustre: server umount lustre-OST0000 complete [ 3476.961370] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3482.542090] Lustre: server umount lustre-MDT0000 complete [ 3484.191245] LustreError: 265197:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786545116 with bad export cookie 14898858763181619235 [ 3484.198771] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3484.201058] LustreError: 265197:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3484.372726] Lustre: server umount lustre-MDT0001 complete [ 3488.574378] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 3489.815772] Key type lgssc unregistered [ 3489.988735] LNet: 268135:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3489.993803] LNetError: 268135:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3490.006623] LNet: Removed LNI 192.168.203.138@tcp [ 3490.463102] Key type .llcrypt unregistered [ 3490.464424] Key type ._llcrypt unregistered [ 3503.423448] Key type ._llcrypt registered [ 3503.425016] Key type .llcrypt registered [ 3503.497864] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 3503.866925] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3503.887834] alg: No test for adler32 (adler32-zlib) [ 3504.799387] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 3504.934662] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 3506.552160] Key type lgssc registered [ 3507.145321] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3511.106404] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3511.116984] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3512.297454] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3514.081735] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3517.799181] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3517.967885] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3519.467520] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3523.011735] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3525.422370] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3528.318374] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3528.532438] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3531.278465] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3534.515881] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3534.968562] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 3535.893496] Lustre: server umount lustre-OST0000 complete [ 3538.401361] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3538.406111] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3538.414516] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3538.914484] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3538.923856] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3543.010560] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3543.014923] Lustre: Skipped 1 previous similar message [ 3543.806400] Lustre: server umount lustre-MDT0000 complete [ 3545.758568] LustreError: 272402:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786545177 with bad export cookie 5076418155223312212 [ 3545.766121] LustreError: 272402:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3545.766255] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3545.962374] Lustre: server umount lustre-MDT0001 complete [ 3551.007589] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 3552.658452] Key type lgssc unregistered [ 3552.842691] LNet: 273158:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3552.847494] LNetError: 273158:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3552.857692] LNet: Removed LNI 192.168.203.138@tcp [ 3553.363172] Key type .llcrypt unregistered [ 3553.365683] Key type ._llcrypt unregistered [ 3560.341268] Key type ._llcrypt registered [ 3560.343029] Key type .llcrypt registered [ 3560.390595] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 3560.758818] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3560.810178] alg: No test for adler32 (adler32-zlib) [ 3561.664683] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 3561.758141] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 3563.336119] Key type lgssc registered [ 3563.784102] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3567.982250] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3567.994776] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3583.469119] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3588.576430] LustreError: 274328:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.138@tcp: failed processing log, type 4: rc = -110 [ 3615.200285] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3617.766271] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3621.123267] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3622.105591] Lustre: server umount lustre-OST0000 complete [ 3629.268190] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 3630.757691] Key type lgssc unregistered [ 3630.899731] LNet: 275959:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3630.905286] LNetError: 275959:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3630.914581] LNet: Removed LNI 192.168.203.138@tcp [ 3631.309716] Key type .llcrypt unregistered [ 3631.311737] Key type ._llcrypt unregistered [ 3647.273897] Key type ._llcrypt registered [ 3647.275376] Key type .llcrypt registered [ 3647.317713] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 3647.612760] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3647.623282] alg: No test for adler32 (adler32-zlib) [ 3648.475811] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 3648.561313] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 3650.152223] Key type lgssc registered [ 3650.480442] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3656.436419] loop0: detected capacity change from 0 to 204800000 [ 3656.549701] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3656.555774] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3656.736797] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3657.229855] loop0: detected capacity change from 0 to 204800000 [ 3657.327528] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3657.334196] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3657.540257] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3658.023731] loop0: detected capacity change from 0 to 204800000 [ 3658.131484] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3658.138262] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3658.205255] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3658.731878] loop0: detected capacity change from 0 to 204800000 [ 3658.843869] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3659.390398] loop0: detected capacity change from 0 to 204800000 [ 3659.405159] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3661.174433] loop0: detected capacity change from 0 to 204800000 [ 3661.192630] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3662.752198] loop0: detected capacity change from 0 to 204800000 [ 3662.768875] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3664.569725] loop0: detected capacity change from 0 to 204800000 [ 3664.585809] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3666.542616] loop0: detected capacity change from 0 to 204800000 [ 3666.582829] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3666.592953] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3666.647858] Lustre: 279348:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3666.651536] Lustre: 279348:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 3666.654892] Lustre: 279348:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3666.658331] Lustre: 279348:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3666.661424] Lustre: 279348:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 3666.664629] Lustre: 279348:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3666.667952] Lustre: 279348:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 3666.671080] Lustre: 279348:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 3666.674351] Lustre: 279348:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 3666.677426] Lustre: 279348:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 3666.680631] Lustre: 279348:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 3666.683593] Lustre: 279348:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 3666.686660] Lustre: 279348:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 3666.690086] Lustre: 279348:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 3669.936655] Lustre: server umount MGS complete [ 3670.359956] loop0: detected capacity change from 0 to 204800000 [ 3670.406855] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3670.622167] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3670.642103] Lustre: lustre-MDT0000: Aborting client recovery [ 3671.027087] loop1: detected capacity change from 0 to 204800000 [ 3671.075409] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3671.132336] Lustre: 279779:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3671.135684] Lustre: 279779:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 3671.140455] Lustre: 279779:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3671.143810] Lustre: 279779:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 3671.147764] Lustre: 279779:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3671.151597] Lustre: 279779:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3671.155348] Lustre: 279779:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 3671.162961] Lustre: Found index 1 for lustre-MDT0001, updating log [ 3671.237581] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3671.265309] Lustre: lustre-MDT0001: Aborting client recovery [ 3671.679366] loop2: detected capacity change from 0 to 204800000 [ 3671.725210] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3671.781822] Lustre: 279860:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3671.785602] Lustre: 279860:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3671.789487] Lustre: 279860:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3671.793463] Lustre: 279860:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3671.796919] Lustre: 279860:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3671.800563] Lustre: 279860:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3671.803853] Lustre: 279860:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3671.868677] Lustre: Found index 0 for lustre-OST0000, updating log [ 3671.882456] Lustre: 279878:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3671.911827] Lustre: lustre-OST0000: Aborting recovery [ 3672.263113] loop3: detected capacity change from 0 to 204800000 [ 3672.300710] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3672.345642] Lustre: 279944:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3672.350093] Lustre: 279944:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3672.357385] Lustre: 279944:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3672.364665] Lustre: 279944:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3672.371839] Lustre: 279944:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3672.378536] Lustre: 279944:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3672.384680] Lustre: 279944:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3672.405166] Lustre: 279950:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3672.425679] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3672.428042] Lustre: Skipped 1 previous similar message [ 3672.434313] Lustre: lustre-OST0001: Aborting recovery [ 3677.670625] LustreError: 279723:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 3680.738197] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3680.738683] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3680.748932] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 3680.755062] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 3681.763506] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 3681.763510] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 3683.302240] LustreError: 279688:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 432us: rc = -2 [ 3697.122147] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3697.122875] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3697.123385] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3697.143099] Lustre: Skipped 3 previous similar messages [ 3701.258286] Lustre: server umount lustre-MDT0000 complete [ 3701.701370] LustreError: 279673:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786545333 with bad export cookie 3411772492469191545 [ 3701.705104] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3701.708829] LustreError: 279673:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3702.241193] LustreError: 280176:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3702.241480] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3702.241976] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3702.241989] Lustre: Skipped 3 previous similar messages [ 3702.252984] LustreError: 280176:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3702.262802] Lustre: Skipped 1 previous similar message [ 3706.272944] LustreError: 279686:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3706.272956] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3706.289366] Lustre: Skipped 2 previous similar messages [ 3711.393242] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3711.393241] LustreError: 279717:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3711.397605] Lustre: Skipped 1 previous similar message [ 3711.412074] LustreError: 279717:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3716.064230] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3716.189983] Lustre: server umount lustre-MDT0001 complete [ 3722.770770] Lustre: server umount lustre-OST0000 complete [ 3729.335563] Lustre: server umount lustre-OST0001 complete [ 3732.853433] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 10:36:04 (1786545364) [ 3745.701440] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_hostid [ 3749.174135] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 3762.999927] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3763.086144] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3763.095452] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3763.132377] Lustre: lustre-MDT0000: new disk, initializing [ 3763.153412] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3763.158862] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3764.354388] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3768.289911] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3768.328202] Lustre: 284007:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3768.353176] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3768.354750] Lustre: Skipped 1 previous similar message [ 3768.392155] Lustre: lustre-MDT0001: new disk, initializing [ 3768.426156] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3768.437267] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3768.440480] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3769.740509] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3773.493860] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3776.128940] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3779.504984] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3779.671853] Lustre: lustre-OST0000: new disk, initializing [ 3779.675133] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3779.680503] Lustre: 285979:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3779.733594] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3780.442123] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3780.449751] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3780.510364] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3782.653730] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3786.672658] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3789.563751] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 3799.601709] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 3804.655163] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 3847.140875] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3847.142844] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3847.144677] LustreError: Skipped 1 previous similar message [ 3847.150883] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3847.155043] Lustre: Skipped 1 previous similar message [ 3849.685682] Lustre: server umount lustre-OST0000 complete [ 3852.258100] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3852.262011] Lustre: Skipped 2 previous similar messages [ 3857.341108] Lustre: server umount lustre-MDT0000 complete [ 3857.377723] LustreError: 284808:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3857.385585] LustreError: 284808:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3858.951158] LustreError: 284001:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786545490 with bad export cookie 3411772492469194247 [ 3858.958309] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3858.959022] LustreError: 284001:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3859.115757] Lustre: server umount lustre-MDT0001 complete [ 3862.640434] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3864.037486] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3866.018893] Lustre: server umount MGS complete [ 3870.589369] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3870.753720] LustreError: 289984:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3870.759806] LustreError: 289997:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 3870.779224] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3872.034142] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3874.899042] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3876.742859] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3879.186315] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3881.100564] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3883.669370] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3885.788621] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3888.307592] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3920.867228] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3920.871613] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3920.874086] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3920.882586] Lustre: Skipped 1 previous similar message [ 3920.890635] Lustre: Skipped 3 previous similar messages [ 3925.798602] Lustre: server umount lustre-OST0000 complete [ 3931.106275] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3933.688148] Lustre: server umount lustre-MDT0000 complete [ 3935.066661] LustreError: 289966:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786545567 with bad export cookie 3411772492469196060 [ 3935.067736] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3935.074321] LustreError: 289966:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3935.212579] Lustre: server umount lustre-MDT0001 complete [ 3938.733379] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 3939.768970] Key type lgssc unregistered [ 3939.896400] LNet: 293947:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3939.899108] LNetError: 293947:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3939.906303] LNet: Removed LNI 192.168.203.138@tcp [ 3940.285703] Key type .llcrypt unregistered [ 3940.288299] Key type ._llcrypt unregistered [ 3955.448389] Key type ._llcrypt registered [ 3955.449603] Key type .llcrypt registered [ 3955.501445] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_hostid [ 3962.846319] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 3963.143793] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3963.151474] alg: No test for adler32 (adler32-zlib) [ 3964.002745] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 3964.088585] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 3965.672187] Key type lgssc registered [ 3966.191630] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3979.852628] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3979.863119] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3980.961899] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3980.974327] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3981.014050] Lustre: lustre-MDT0000: new disk, initializing [ 3981.035609] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3981.042418] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3982.790487] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3987.790586] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3987.853303] Lustre: 297874:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3987.889110] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3987.892472] Lustre: Skipped 1 previous similar message [ 3987.938189] Lustre: lustre-MDT0001: new disk, initializing [ 3987.970837] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3987.983969] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3987.990134] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3989.504483] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 3993.701972] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3995.836361] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3998.615436] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3998.780150] Lustre: lustre-OST0000: new disk, initializing [ 3998.784083] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3998.789612] Lustre: 299842:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3998.840170] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4001.055639] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4005.497627] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4006.925047] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4006.934252] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4006.991930] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4014.843888] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4017.591869] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4024.841922] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4071.397266] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4071.399784] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4071.405028] Lustre: Skipped 1 previous similar message [ 4074.723096] Lustre: server umount lustre-OST0000 complete [ 4076.514617] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4076.515559] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4076.524487] Lustre: Skipped 1 previous similar message [ 4076.528766] Lustre: Skipped 1 previous similar message [ 4081.633731] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4081.639350] Lustre: Skipped 1 previous similar message [ 4082.527015] Lustre: server umount lustre-MDT0000 complete [ 4084.020515] LustreError: 297866:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786545715 with bad export cookie 3836786000878669278 [ 4084.027928] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4084.030177] LustreError: 297866:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4084.186768] Lustre: server umount lustre-MDT0001 complete [ 4088.344637] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4090.122559] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4092.316257] Lustre: server umount MGS complete [ 4096.473961] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4096.679714] LustreError: 303715:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4096.715634] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4098.422604] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4101.643618] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4101.782756] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4103.275103] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4106.287246] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4108.031958] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4110.497389] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4110.629561] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4112.730946] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4115.111303] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4147.681765] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4147.684162] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4147.684884] LustreError: Skipped 1 previous similar message [ 4147.688490] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4147.691469] Lustre: Skipped 1 previous similar message [ 4149.526888] Lustre: server umount lustre-OST0000 complete [ 4152.801570] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4152.802417] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4152.804734] Lustre: Skipped 2 previous similar messages [ 4152.807977] Lustre: Skipped 1 previous similar message [ 4163.041109] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4163.042899] Lustre: Skipped 3 previous similar messages [ 4165.088138] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4165.158434] Lustre: server umount lustre-MDT0000 complete [ 4166.420510] LustreError: 303683:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786545798 with bad export cookie 3836786000878671091 [ 4166.422889] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4166.424761] LustreError: 303683:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4166.565939] Lustre: server umount lustre-MDT0001 complete [ 4170.349610] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 4171.393469] Key type lgssc unregistered [ 4171.527440] LNet: 307674:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4171.530802] LNetError: 307674:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4171.536402] LNet: Removed LNI 192.168.203.138@tcp [ 4171.842520] Key type .llcrypt unregistered [ 4171.843937] Key type ._llcrypt unregistered [ 4178.627838] Key type ._llcrypt registered [ 4178.628797] Key type .llcrypt registered [ 4178.674163] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_hostid [ 4184.907068] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 4185.334337] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4185.341476] alg: No test for adler32 (adler32-zlib) [ 4186.197940] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 4186.293873] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 4187.880225] Key type lgssc registered [ 4188.354818] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4198.441547] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 4204.330750] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4204.338835] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4205.440420] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4205.451790] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4205.490466] Lustre: lustre-MDT0000: new disk, initializing [ 4205.511206] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4205.516507] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4206.708414] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4210.631950] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4210.735364] Lustre: lustre-OST0000: new disk, initializing [ 4210.737700] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4210.740238] Lustre: Skipped 1 previous similar message [ 4210.742780] Lustre: 311698:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4210.764656] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4211.650813] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4211.654531] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4211.662479] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4212.574456] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4441.059162] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4441.072837] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4446.132865] Lustre: server umount lustre-MDT0000 complete [ 4448.127840] LustreError: 310883:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786546080 with bad export cookie 13297556561813620050 [ 4448.131573] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4448.135963] LustreError: 310883:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4448.242426] Lustre: server umount lustre-OST0000 complete [ 4460.594662] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_hostid [ 4465.487619] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 4482.787908] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4482.940273] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4482.960518] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4483.015650] Lustre: lustre-MDT0000: new disk, initializing [ 4483.047866] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4483.059891] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4484.950254] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4490.363847] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4490.426292] Lustre: 315933:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4490.453113] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4490.455096] Lustre: Skipped 1 previous similar message [ 4490.497806] Lustre: lustre-MDT0001: new disk, initializing [ 4490.537223] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4490.553839] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4490.557991] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4492.146822] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4496.893349] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4499.593056] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4502.959593] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4503.136922] Lustre: lustre-OST0000: new disk, initializing [ 4503.138818] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4503.142477] Lustre: 317900:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4503.188062] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4506.002614] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4508.706225] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4508.713118] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4508.771003] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4510.334481] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4512.433714] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4512.583624] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4514.694424] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4514.798351] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4518.887153] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4518.889448] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4518.893982] LustreError: Skipped 1 previous similar message [ 4518.903736] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4518.909739] Lustre: Skipped 1 previous similar message [ 4524.006162] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4524.011381] Lustre: Skipped 1 previous similar message [ 4528.038748] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4528.043047] Lustre: Skipped 1 previous similar message [ 4530.144158] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4530.220512] Lustre: server umount lustre-OST0000 complete [ 4533.219273] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4533.220762] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4533.227505] Lustre: Skipped 2 previous similar messages [ 4533.235253] Lustre: Skipped 1 previous similar message [ 4538.165321] Lustre: server umount lustre-MDT0000 complete [ 4539.831720] LustreError: 315925:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786546171 with bad export cookie 13297556561816560827 [ 4539.838490] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4539.989983] Lustre: server umount lustre-MDT0001 complete [ 4541.713360] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 4542.581843] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 10:49:34 (1786546174) [ 4545.918865] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4546.135382] LustreError: 320213:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4546.143765] LustreError: 320213:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 4546.168540] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4548.024762] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4551.650106] LustreError: 320214:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4552.230981] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4554.110000] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4557.636599] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4560.170360] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4563.472495] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4563.640791] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4563.644420] Lustre: Skipped 1 previous similar message [ 4566.243785] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4569.001809] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4571.702907] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4571.856432] Lustre: lustre-OST0001: new disk, initializing [ 4571.859268] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4571.863367] Lustre: 323057:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4573.191313] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4573.194739] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4573.211612] LustreError: 323239:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4574.065866] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4574.245483] LustreError: 323243:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4574.248729] LustreError: 323243:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 4575.267572] LustreError: 323239:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4575.270953] LustreError: 323239:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 4577.319083] LustreError: 323243:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4577.322559] LustreError: 323243:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 4578.035132] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4578.112522] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4580.123469] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff954bc2f62000.ost_server_uuid 50 [ 4580.847108] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff954bc2f62000.ost_server_uuid in FULL state after 0 sec [ 4581.415184] LustreError: 323243:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4581.418522] LustreError: 323243:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 4583.088441] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff954bc2f62000.ost_server_uuid 50 [ 4583.693641] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff954bc2f62000.ost_server_uuid in FULL state after 0 sec [ 4584.489800] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4597.245965] Lustre: lustre-OST0001: Not available for connect from 192.168.203.38@tcp (stopping) [ 4597.736408] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4601.554986] Lustre: server umount lustre-OST0001 complete [ 4602.354687] LustreError: 322034:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.203.38@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4607.973197] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4607.974975] LustreError: 322035:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4607.975626] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4607.975640] Lustre: Skipped 1 previous similar message [ 4607.978548] LustreError: Skipped 1 previous similar message [ 4607.989747] LustreError: 322035:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 4609.772652] Lustre: server umount lustre-OST0000 complete [ 4617.692734] Lustre: server umount lustre-MDT0000 complete [ 4618.210048] LustreError: 320213:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4618.219858] LustreError: 320213:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4619.215040] LustreError: 323056:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786546251 with bad export cookie 13297556561816561695 [ 4619.219436] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4619.220330] LustreError: 323056:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 4619.360803] Lustre: server umount lustre-MDT0001 complete [ 4623.924062] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 4625.383184] Key type lgssc unregistered [ 4625.551769] LNet: 325742:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4625.557218] LNetError: 325742:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4625.567460] LNet: Removed LNI 192.168.203.138@tcp [ 4626.056935] Key type .llcrypt unregistered [ 4626.059320] Key type ._llcrypt unregistered [ 4637.799210] Key type ._llcrypt registered [ 4637.800980] Key type .llcrypt registered [ 4637.856705] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 4638.270617] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4638.278822] alg: No test for adler32 (adler32-zlib) [ 4639.145375] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 4639.243815] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 4640.824256] Key type lgssc registered [ 4641.447725] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4645.059493] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4645.072504] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4646.273669] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4648.057101] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4651.644799] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4655.184645] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4655.321609] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4656.955876] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4661.088495] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4661.327116] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4663.974447] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4665.450629] LustreError: 328817:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4665.457712] LustreError: 328817:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4666.699593] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4669.237205] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4669.314212] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4671.067183] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4671.405618] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 4671.407416] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 4673.315544] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4675.824309] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 4675.900871] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4753.382184] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4753.387963] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4753.388147] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4758.500380] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4758.503032] Lustre: Skipped 2 previous similar messages [ 4763.620330] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4763.625123] Lustre: Skipped 1 previous similar message [ 4765.152161] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4765.190622] Lustre: server umount lustre-OST0001 complete [ 4768.225126] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4768.227478] LustreError: Skipped 1 previous similar message [ 4768.228711] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4768.232062] Lustre: Skipped 1 previous similar message [ 4768.233662] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4768.744922] LustreError: 328820:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4768.749682] LustreError: 328820:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4773.859589] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4773.861536] LustreError: 328820:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4773.865419] Lustre: Skipped 2 previous similar messages [ 4773.876239] LustreError: 328820:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4778.980515] LustreError: 328817:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4778.993627] LustreError: 328817:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4781.024169] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4781.106602] Lustre: server umount lustre-OST0000 complete [ 4783.585680] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4783.591637] LustreError: Skipped 1 previous similar message [ 4783.594775] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4783.603085] Lustre: Skipped 1 previous similar message [ 4783.606498] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4783.610359] Lustre: Skipped 2 previous similar messages [ 4789.117211] Lustre: server umount lustre-MDT0000 complete [ 4789.217429] LustreError: 327138:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4789.226429] LustreError: 327138:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4790.846540] LustreError: 327121:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786546422 with bad export cookie 5720319643951339863 [ 4790.849822] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4790.851639] LustreError: 327121:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4790.997683] Lustre: server umount lustre-MDT0001 complete [ 4795.705700] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 4796.819985] Key type lgssc unregistered [ 4796.939319] LNet: 332078:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4796.942446] LNetError: 332078:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4796.950389] LNet: Removed LNI 192.168.203.138@tcp [ 4797.322481] Key type .llcrypt unregistered [ 4797.324549] Key type ._llcrypt unregistered [ 4818.847039] Key type ._llcrypt registered [ 4818.848458] Key type .llcrypt registered [ 4818.892633] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 4819.332427] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4819.342891] alg: No test for adler32 (adler32-zlib) [ 4820.190690] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 4820.271309] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 4821.848230] Key type lgssc registered [ 4822.464108] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4826.639558] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4826.645977] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4827.806044] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4829.417857] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4833.274842] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4833.415521] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4835.010275] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4836.242309] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 4839.366697] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4839.594344] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4841.998373] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4843.688653] LustreError: 336373:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4843.698525] LustreError: 336373:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4845.852578] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4845.986380] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4848.794599] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4852.206397] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 4852.208286] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 4858.117602] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4865.386688] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 4872.186991] Lustre: server umount lustre-MDT0000 complete [ 4872.675645] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4872.677776] LustreError: 334337:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4872.682414] Lustre: Skipped 2 previous similar messages [ 4872.691961] LustreError: 334337:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 4873.838322] LustreError: 337351:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786546505 with bad export cookie 2754060617290070957 [ 4873.847035] LustreError: 337351:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4873.848815] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4876.047889] Lustre: server umount lustre-MDT0001 complete [ 4888.080549] Lustre: server umount lustre-OST0000 complete [ 4900.016759] Lustre: server umount lustre-OST0001 complete [ 4906.032811] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 4909.771556] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4909.909877] LustreError: 339601:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4909.925633] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4911.523047] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4914.651734] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4916.238563] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4917.533307] Lustre: 340743:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4920.689231] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4920.853743] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4920.856161] Lustre: Skipped 1 previous similar message [ 4922.601679] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4924.964047] LustreError: 341096:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4925.157403] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4926.937865] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 4934.634916] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 4934.635456] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 4940.821198] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4942.139059] Lustre: 342634:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4944.868027] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4944.868603] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4944.871748] Lustre: Skipped 2 previous similar messages [ 4949.373405] Lustre: server umount lustre-MDT0000 complete [ 4949.987591] LustreError: 340334:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4949.997789] LustreError: 340334:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 4950.978572] LustreError: 339582:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786546582 with bad export cookie 2754060617290072616 [ 4950.982603] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4950.986228] LustreError: 339582:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4951.144900] Lustre: server umount lustre-MDT0001 complete [ 4962.306723] Lustre: server umount lustre-OST0000 complete [ 4974.109472] Lustre: server umount lustre-OST0001 complete [ 4975.315527] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 4975.892651] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 4976.481602] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 10:56:48 (1786546608) [ 4983.689789] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 4987.153397] loop0: detected capacity change from 0 to 1099511627776 [ 4987.312175] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 4987.314985] *** VALIDATE xfs *** [ 4987.320598] XFS (loop0): Mounting V5 Filesystem [ 4987.329183] XFS (loop0): Ending clean mount [ 4987.736714] loop1: detected capacity change from 0 to 18691697672192 [ 5008.688097] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_hostid [ 5013.105862] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 5028.330461] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5028.448895] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5028.466469] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5028.524996] Lustre: lustre-MDT0000: new disk, initializing [ 5028.563638] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5028.569114] Lustre: Skipped 1 previous similar message [ 5028.579962] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5030.079162] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 5034.654073] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5034.695106] Lustre: 347746:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5034.721947] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5034.723885] Lustre: Skipped 1 previous similar message [ 5034.760947] Lustre: lustre-MDT0001: new disk, initializing [ 5034.815293] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5034.819628] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5036.348682] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 5040.827432] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5043.500159] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5046.813313] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5046.985313] Lustre: lustre-OST0000: new disk, initializing [ 5046.987863] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5046.991533] Lustre: 349714:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5048.970887] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5048.975171] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5049.032612] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5049.527802] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 5054.484428] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5056.376072] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5056.470576] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5058.334838] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5058.430528] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5059.557338] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5059.558326] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5059.559685] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5059.559694] Lustre: Skipped 3 previous similar messages [ 5059.563369] LustreError: Skipped 1 previous similar message [ 5059.579350] Lustre: Skipped 3 previous similar messages [ 5064.678648] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5064.680774] Lustre: Skipped 2 previous similar messages [ 5069.802555] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5069.806803] Lustre: Skipped 1 previous similar message [ 5073.376163] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5073.452526] Lustre: server umount lustre-OST0000 complete [ 5075.937688] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5075.941875] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5075.948740] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5080.035966] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5081.473871] Lustre: server umount lustre-MDT0000 complete [ 5083.004727] LustreError: 347782:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786546714 with bad export cookie 2754060617290074149 [ 5083.008188] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5083.011286] LustreError: 347782:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5083.124016] Lustre: server umount lustre-MDT0001 complete [ 5086.455734] XFS (loop0): Unmounting Filesystem [ 5088.472595] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 10:58:40 (1786546720) [ 5091.217064] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5091.428720] LustreError: 352273:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5091.462368] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5091.465770] Lustre: Skipped 2 previous similar messages [ 5093.260643] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 5097.222308] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5099.258059] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 5102.742370] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5104.965575] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5107.366013] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5109.554976] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 5112.733359] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5122.026382] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5122.036867] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5122.037100] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5122.044620] Lustre: Skipped 3 previous similar messages [ 5123.157344] Lustre: server umount lustre-OST0000 complete [ 5126.706576] Lustre: server umount lustre-MDT0000 complete [ 5127.141456] LustreError: 352273:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5127.151644] LustreError: 352273:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5128.241246] LustreError: 352254:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786546760 with bad export cookie 2754060617290075031 [ 5128.244841] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5128.246267] LustreError: 352254:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5128.370611] Lustre: server umount lustre-MDT0001 complete [ 5131.956852] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 5132.929872] Key type lgssc unregistered [ 5133.052374] LNet: 355901:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5133.056263] LNetError: 355901:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5133.065241] LNet: Removed LNI 192.168.203.138@tcp [ 5133.371140] Key type .llcrypt unregistered [ 5133.372216] Key type ._llcrypt unregistered [ 5143.475923] Key type ._llcrypt registered [ 5143.477077] Key type .llcrypt registered [ 5143.519717] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 5143.860587] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5143.875309] alg: No test for adler32 (adler32-zlib) [ 5144.719183] Lustre: Lustre: Build Version: 2.17.57_1_g9701683 [ 5144.806986] LNet: Added LNI 192.168.203.138@tcp [8/256/0/180] [ 5146.384142] Key type lgssc registered [ 5146.726612] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5149.357984] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5149.370619] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5150.556956] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5152.166114] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 5155.716896] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5155.909522] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5157.374070] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 5159.671460] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5161.495476] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5163.547727] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5163.651813] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5165.389170] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 5167.787201] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5181.410910] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5181.417718] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5181.420592] Lustre: Skipped 1 previous similar message [ 5183.486949] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.38@tcp (stopping) [ 5186.533868] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5186.538052] Lustre: Skipped 1 previous similar message [ 5187.586812] Lustre: server umount lustre-MDT0001 complete [ 5188.595724] LustreError: 357449:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.38@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5188.608457] LustreError: 357449:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5191.653710] LustreError: 357449:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5191.665520] LustreError: 357449:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5192.974605] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5193.033750] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5193.037102] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5193.061698] Lustre: 357444:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5193.213225] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5195.124173] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 5198.320575] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5202.408719] LustreError: 361232:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5202.413674] LustreError: 361232:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5202.419595] LustreError: 357446:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5202.457752] LustreError: 361236:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.138@tcp: cfg command failed: rc = -17 [ 5202.465754] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 5202.465754] [ 5500.389646] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5500.390511] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5500.399104] Lustre: Skipped 1 previous similar message [ 5500.407714] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5500.413450] Lustre: Skipped 1 previous similar message [ 5503.785377] Lustre: server umount lustre-MDT0001 complete [ 5505.507956] LustreError: 357454:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5505.518690] LustreError: 357454:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5507.886353] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5507.949888] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5507.954310] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5507.977141] Lustre: 361233:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5508.111903] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5509.724297] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 5513.198343] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5513.703078] LustreError: 372868:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5513.705840] LustreError: 372868:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5513.708590] LustreError: 357446:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5513.710663] LustreError: 357446:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 5513.727608] LustreError: 372871:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [ 5513.730489] LustreError: 372871:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [ 5513.732517] LustreError: 372871:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.138@tcp: cfg command failed: rc = -17 [ 5513.736420] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.138@tcp [ 5513.736420] [ 5813.249734] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.38@tcp (stopping) [ 5813.254545] Lustre: Skipped 1 previous similar message [ 5815.270178] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5815.272630] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5818.426648] Lustre: server umount lustre-MDT0001 complete [ 5820.386725] LustreError: 357453:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5820.397369] LustreError: 357453:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5823.249696] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5823.313333] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5823.317410] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5823.341359] Lustre: 357443:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5823.507399] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 5825.204361] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 5828.585315] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5830.634547] LustreError: 384593:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5830.640288] LustreError: 384593:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5830.646570] LustreError: 357446:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5830.651143] LustreError: 357446:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 5830.681559] LustreError: 384596:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 5830.692387] LustreError: 384596:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.138@tcp: cfg command failed: rc = -114 [ 5830.698812] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 5830.698812] [ 6140.905227] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6140.909646] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6140.913677] Lustre: Skipped 3 previous similar messages [ 6140.914126] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6140.918040] Lustre: Skipped 5 previous similar messages [ 6154.208190] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6154.321192] Lustre: server umount lustre-MDT0000 complete [ 6155.913720] LustreError: 357436:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786547787 with bad export cookie 2975464547917500415 [ 6155.921187] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6155.921726] LustreError: 357436:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6156.257677] LustreError: 358749:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6156.258331] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6160.290208] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6160.296791] Lustre: Skipped 7 previous similar messages [ 6162.136659] Lustre: server umount lustre-MDT0001 complete [ 6164.073528] Lustre: server umount lustre-OST0000 complete [ 6168.050957] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_hostid [ 6172.308978] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 6189.041356] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 11:17:00 (1786547820) [ 6192.215191] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6192.326909] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6192.343298] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6192.400392] Lustre: lustre-MDT0000: new disk, initializing [ 6192.433816] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6192.443806] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6194.252455] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 6200.059325] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6200.128419] Lustre: 398208:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6200.166309] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6200.169623] Lustre: Skipped 1 previous similar message [ 6200.214616] Lustre: lustre-MDT0001: new disk, initializing [ 6200.263355] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6200.282411] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6200.285809] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6201.912345] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 6207.037852] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6209.790376] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6213.133987] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6213.326895] Lustre: lustre-OST0000: new disk, initializing [ 6213.330231] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6213.335141] Lustre: 400177:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6213.388345] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6215.964423] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_default_debug -1 all [ 6220.833215] Lustre: DEBUG MARKER: oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6222.864133] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6222.871514] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6222.936888] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6231.009494] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6231.014768] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6231.023733] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6235.369360] Lustre: server umount lustre-MDT0000 complete [ 6236.587232] LustreError: 399003:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786547868 with bad export cookie 2975464547917502354 [ 6236.590337] LustreError: 399003:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6236.590541] LustreError: 399015:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 6236.590639] LustreError: MGC192.168.203.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6236.698480] Lustre: server umount lustre-MDT0001 complete [ 6248.695131] Lustre: server umount lustre-OST0000 complete [ 6261.412988] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing set_hostid [ 6265.919190] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing load_modules_local [ 6279.703807] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 6146 sec ========== 11:18:31 (1786547911) [ 6280.434645] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 11:18:32 (1786547912) === [ 6281.783657] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 11:18:33 (1786547913) === [ 6300.003105] Lustre: DEBUG MARKER: oleg338-server.virtnet: executing unload_modules_local [ 6301.548995] Key type lgssc unregistered [ 6301.738845] LNet: 406534:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6301.744341] LNetError: 406534:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6301.756767] LNet: Removed LNI 192.168.203.138@tcp [ 6302.265869] Key type .llcrypt unregistered [ 6302.268595] Key type ._llcrypt unregistered