[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 534636831 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.003306] x2apic enabled [ 0.004009] Switched APIC routing to physical x2apic. [ 0.005014] kvm-guest: setup PV IPIs [ 0.008292] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009020] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010012] pid_max: default: 32768 minimum: 301 [ 0.011130] LSM: Security Framework initializing [ 0.012048] Yama: becoming mindful. [ 0.013038] SELinux: Initializing. [ 0.014075] *** VALIDATE selinux *** [ 0.022641] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026798] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027138] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029105] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030102] *** VALIDATE tmpfs *** [ 0.031420] *** VALIDATE proc *** [ 0.033134] *** VALIDATE cgroup *** [ 0.034010] *** VALIDATE cgroup2 *** [ 0.035262] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036155] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038030] Spectre V2 : User space: Vulnerable [ 0.039009] Speculative Store Bypass: Vulnerable [ 0.041838] debug: unmapping init [mem 0xffffffff89259000-0xffffffff89260fff] [ 0.043161] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044612] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045022] ... version: 2 [ 0.046015] ... bit width: 48 [ 0.047011] ... generic registers: 4 [ 0.048012] ... value mask: 0000ffffffffffff [ 0.049013] ... max period: 00007fffffffffff [ 0.050012] ... fixed-purpose events: 3 [ 0.051009] ... event mask: 000000070000000f [ 0.052297] rcu: Hierarchical SRCU implementation. [ 0.054362] smp: Bringing up secondary CPUs ... [ 0.055569] x86: Booting SMP configuration: [ 0.056025] .... node #0, CPUs: #1 #2 #3 [ 0.059566] smp: Brought up 1 node, 4 CPUs [ 0.061013] smpboot: Max logical packages: 1 [ 0.062016] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.154395] node 0 deferred pages initialised in 89ms [ 0.159112] devtmpfs: initialized [ 0.160471] x86/mm: Memory block size: 128MB [ 0.164405] gcov: version magic: 0x41383552 [ 0.168308] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.172094] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.175295] pinctrl core: initialized pinctrl subsystem [ 0.177175] [ 0.177737] ************************************************************* [ 0.180013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.183012] ** ** [ 0.185023] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.188014] ** ** [ 0.191012] ** This means that this kernel is built to expose internal ** [ 0.193012] ** IOMMU data structures, which may compromise security on ** [ 0.196013] ** your system. ** [ 0.198010] ** ** [ 0.201012] ** If you see this message and you are not debugging the ** [ 0.203012] ** kernel, report this immediately to your vendor! ** [ 0.205011] ** ** [ 0.208013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.210011] ************************************************************* [ 0.214721] NET: Registered protocol family 16 [ 0.216402] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.218064] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.221062] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.225049] cpuidle: using governor menu [ 0.226744] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.229594] PCI: Using configuration type 1 for base access [ 0.231199] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.241085] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.242032] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.244084] cryptd: max_cpu_qlen set to 1000 [ 0.246326] ACPI: Added _OSI(Module Device) [ 0.247018] ACPI: Added _OSI(Processor Device) [ 0.248000] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.250023] ACPI: Added _OSI(Processor Aggregator Device) [ 0.255327] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.263668] ACPI: Interpreter enabled [ 0.265088] ACPI: PM: (supports S0 S3 S4 S5) [ 0.267018] ACPI: Using IOAPIC for interrupt routing [ 0.269125] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.272387] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.284712] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.287038] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.288018] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.291065] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.296403] acpiphp: Slot [2] registered [ 0.298102] acpiphp: Slot [5] registered [ 0.299106] acpiphp: Slot [6] registered [ 0.300109] acpiphp: Slot [7] registered [ 0.302078] acpiphp: Slot [8] registered [ 0.303110] acpiphp: Slot [9] registered [ 0.304113] acpiphp: Slot [10] registered [ 0.306169] acpiphp: Slot [3] registered [ 0.307081] acpiphp: Slot [4] registered [ 0.308112] acpiphp: Slot [11] registered [ 0.310112] acpiphp: Slot [12] registered [ 0.311106] acpiphp: Slot [13] registered [ 0.313094] acpiphp: Slot [14] registered [ 0.314113] acpiphp: Slot [15] registered [ 0.316124] acpiphp: Slot [16] registered [ 0.317083] acpiphp: Slot [17] registered [ 0.319098] acpiphp: Slot [18] registered [ 0.320123] acpiphp: Slot [19] registered [ 0.322128] acpiphp: Slot [20] registered [ 0.323091] acpiphp: Slot [21] registered [ 0.325100] acpiphp: Slot [22] registered [ 0.326086] acpiphp: Slot [23] registered [ 0.327162] acpiphp: Slot [24] registered [ 0.329108] acpiphp: Slot [25] registered [ 0.331104] acpiphp: Slot [26] registered [ 0.333141] acpiphp: Slot [27] registered [ 0.334113] acpiphp: Slot [28] registered [ 0.336101] acpiphp: Slot [29] registered [ 0.338093] acpiphp: Slot [30] registered [ 0.339105] acpiphp: Slot [31] registered [ 0.341067] PCI host bridge to bus 0000:00 [ 0.343026] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.345032] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.348026] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.350023] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.353024] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.356024] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.358160] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.361063] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.364051] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.375022] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.381063] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.384017] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.386018] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.388019] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.391835] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.395787] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.398040] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.401846] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.408017] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.420868] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.425973] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.431536] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.440014] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.446016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.462015] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.471547] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.481015] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.490015] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.505015] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.514743] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.524024] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.532020] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.551016] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.561053] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.570014] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.576026] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.591016] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.600000] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.607014] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.615014] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.627018] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.637011] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.643856] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.651015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.686017] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.695642] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.699060] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.702342] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.704378] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.707801] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.712210] iommu: Default domain type: Passthrough [ 0.713560] SCSI subsystem initialized [ 0.714216] ACPI: bus type USB registered [ 0.716162] usbcore: registered new interface driver usbfs [ 0.718127] usbcore: registered new interface driver hub [ 0.720123] usbcore: registered new device driver usb [ 0.723233] pps_core: LinuxPPS API ver. 1 registered [ 0.724018] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.727158] PTP clock support registered [ 0.730137] EDAC MC: Ver: 3.0.0 [ 0.732152] PCI: Using ACPI for IRQ routing [ 0.733696] NetLabel: Initializing [ 0.735028] NetLabel: domain hash size = 128 [ 0.736016] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.737000] NetLabel: unlabeled traffic allowed by default [ 0.739055] vgaarb: loaded [ 0.740254] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.742022] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.749322] clocksource: Switched to clocksource kvm-clock [ 0.854250] VFS: Disk quotas dquot_6.6.0 [ 0.856303] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.860272] *** VALIDATE ramfs *** [ 0.862249] *** VALIDATE hugetlbfs *** [ 0.864531] pnp: PnP ACPI init [ 0.867323] pnp: PnP ACPI: found 6 devices [ 0.883810] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.889140] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.892431] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.895116] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.898894] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.902029] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.905054] NET: Registered protocol family 2 [ 0.907313] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.912166] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.916124] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.921516] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.924852] TCP: Hash tables configured (established 65536 bind 65536) [ 0.927676] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.931024] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.934086] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.937267] NET: Registered protocol family 1 [ 0.939600] RPC: Registered named UNIX socket transport module. [ 0.941750] RPC: Registered udp transport module. [ 0.943751] RPC: Registered tcp transport module. [ 0.945543] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.947735] NET: Registered protocol family 44 [ 0.948962] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.950551] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.952059] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.953686] PCI: CLS 0 bytes, default 64 [ 0.954770] Unpacking initramfs... [ 2.383245] debug: unmapping init [mem 0xffff96473cc54000-0xffff96473ffbffff] [ 2.390178] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.392520] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.397474] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.955033] Initialise system trusted keyrings [ 2.957276] Key type blacklist registered [ 2.961618] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.973718] zbud: loaded [ 2.977041] *** VALIDATE nfs *** [ 2.978361] *** VALIDATE nfs4 *** [ 2.980038] pstore: using deflate compression [ 2.984262] Platform Keyring initialized [ 3.093628] NET: Registered protocol family 38 [ 3.095706] Key type asymmetric registered [ 3.097461] Asymmetric key parser 'x509' registered [ 3.099544] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.103120] io scheduler mq-deadline registered [ 3.104967] io scheduler kyber registered [ 3.106589] io scheduler bfq registered [ 3.108803] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.112327] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.115218] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.118282] ACPI: Power Button [PWRF] [ 3.123708] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.130410] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.148122] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.155277] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.167759] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.194848] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.225078] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.229942] Non-volatile memory driver v1.3 [ 3.231838] Linux agpgart interface v0.103 [ 3.260381] virtio_blk virtio1: [vda] 146864 512-byte logical blocks (75.2 MB/71.7 MiB) [ 3.262623] vda: detected capacity change from 0 to 75194368 [ 3.277307] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.279803] vdb: detected capacity change from 0 to 1073741824 [ 3.299114] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.302963] vdc: detected capacity change from 0 to 2621440000 [ 3.326464] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.329273] vdd: detected capacity change from 0 to 2621440000 [ 3.343218] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.346432] vde: detected capacity change from 0 to 4294967296 [ 3.363645] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.366893] vdf: detected capacity change from 0 to 4294967296 [ 3.379777] libphy: Fixed MDIO Bus: probed [ 3.385137] usbcore: registered new interface driver usbserial_generic [ 3.387424] usbserial: USB Serial support registered for generic [ 3.389326] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.392990] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.394627] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.396858] mousedev: PS/2 mouse device common for all mice [ 3.399374] rtc_cmos 00:05: RTC can wake from S4 [ 3.402990] rtc_cmos 00:05: registered as rtc0 [ 3.404043] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.405415] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.411613] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.413924] intel_pstate: CPU model not supported [ 3.418174] hid: raw HID events driver (C) Jiri Kosina [ 3.419761] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.420356] usbcore: registered new interface driver usbhid [ 3.425814] usbhid: USB HID core driver [ 3.427024] drop_monitor: Initializing network drop monitor service [ 3.428770] Initializing XFRM netlink socket [ 3.430411] NET: Registered protocol family 10 [ 3.433260] Segment Routing with IPv6 [ 3.434799] NET: Registered protocol family 17 [ 3.437054] mpls_gso: MPLS GSO support [ 3.442863] RAS: Correctable Errors collector initialized. [ 3.444934] AVX version of gcm_enc/dec engaged. [ 3.446475] AES CTR mode by8 optimization enabled [ 3.526408] sched_clock: Marking stable (3526374755, 0)->(4414113078, -887738323) [ 3.529599] registered taskstats version 1 [ 3.531674] Loading compiled-in X.509 certificates [ 3.533717] zswap: loaded using pool lzo/zbud [ 3.558410] Key type big_key registered [ 3.571403] Key type encrypted registered [ 3.572952] ima: No TPM chip found, activating TPM-bypass! [ 3.575052] ima: Allocated hash algorithm: sha1 [ 3.576631] ima: No architecture policies found [ 3.578312] evm: Initialising EVM extended attributes: [ 3.580282] evm: security.selinux [ 3.581442] evm: security.ima [ 3.582476] evm: security.capability [ 3.583799] evm: HMAC attrs: 0x1 [ 3.586175] rtc_cmos 00:05: setting system clock to 2026-08-27 15:57:53 UTC (1787846273) [ 3.592308] debug: unmapping init [mem 0xffffffff8a203000-0xffffffff8a3fffff] [ 3.595923] debug: unmapping init [mem 0xffffffff88f82000-0xffffffff89258fff] [ 3.605105] Write protecting the kernel read-only data: 28672k [ 3.609250] debug: unmapping init [mem 0xffffffff87603000-0xffffffff877fffff] [ 3.612372] debug: unmapping init [mem 0xffffffff87f14000-0xffffffff87ffffff] [ 3.651105] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.660194] systemd[1]: Detected virtualization kvm. [ 3.662353] systemd[1]: Detected architecture x86-64. [ 3.664239] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.688797] systemd[1]: No hostname configured. [ 3.691111] systemd[1]: Set hostname to . [ 3.692915] random: systemd: uninitialized urandom read (16 bytes read) [ 3.694875] systemd[1]: Initializing machine ID from random generator. [ 3.842842] random: systemd: uninitialized urandom read (16 bytes read) [ 3.844964] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 3.849600] random: systemd: uninitialized urandom read (16 bytes read) [ 3.852515] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 3.856701] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Swap. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Paths. [ OK ] Reached target Slices. [ OK ] Listening on Journal Socket. [ OK ] Reached target Sockets. Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... Starting Create Volatile Files and Directories... [ OK ] Started Memstrack Anylazing Service. Starting Journal Service... Starting Setup Virtual Console... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.514326] device-mapper: uevent: version 1.0.3 [ 4.516686] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System...[ 5.069846] random: fast init done [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.251593] virtio_net virtio0 ens2: renamed from eth0 [ 5.439342] scsi host0: ata_piix [ 5.475241] scsi host1: ata_piix [ 5.476859] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.480940] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.654520] dracut-initqueue[586]: RTNETLINK answers: File exists [ 10.089384] random: crng init done [ 10.090916] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.504478] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Swap. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.646347] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.923239] SELinux: Disabled at runtime. [ 11.984393] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.994405] systemd[1]: Detected virtualization kvm. [ 11.996304] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.477606] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.481197] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.485546] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.491288] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.494831] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.501996] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.505946] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Mounting POSIX Message Queue File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Listening on Process Core Dump Socket. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on udev Control Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... Starting Apply Kernel Variables... [ 12.609080] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Created slice system-getty.slice. [ OK ] Reached target rpc_pipefs.target. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Mounting Huge Pages File System... Mounting Kernel Debug File System... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Starting Remount Root and Kernel File Systems... [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 12.919426] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.240288] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.249454] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.389435] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.407306] EDAC sbridge: Ver: 1.1.2 [ 14.952347] Key type dns_resolver registered [ 15.263666] NFS: Registering the id_resolver key type [ 15.265934] Key type id_resolver registered [ 15.267530] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg650-server login: [ 40.553351] libcfs: loading out-of-tree module taints kernel. [ 40.586031] Key type ._llcrypt registered [ 40.587790] Key type .llcrypt registered [ 40.652839] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_hostid [ 49.124312] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 49.808394] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 49.816098] alg: No test for adler32 (adler32-zlib) [ 50.873700] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [ 51.240377] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [ 52.887295] Key type lgssc registered [ 53.708349] hrtimer: interrupt took 3810309 ns [ 54.755801] Lustre: Echo OBD driver; http://www.lustre.org/ [ 73.643695] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 111.628144] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 125.207359] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 125.256313] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 126.536681] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 126.594517] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 126.693725] Lustre: lustre-MDT0000: new disk, initializing [ 126.774612] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 126.796507] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 131.018422] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 144.743495] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 144.852572] Lustre: 6478:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 144.878677] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 144.882631] Lustre: Skipped 1 previous similar message [ 144.988107] Lustre: lustre-MDT0001: new disk, initializing [ 145.041900] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 145.073567] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 145.089499] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 149.515616] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 154.300451] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 164.507758] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 164.877868] Lustre: lustre-OST0000: new disk, initializing [ 164.881548] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 164.890093] Lustre: 8413:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 164.941540] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 169.485459] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 169.499239] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 169.590089] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 171.507147] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 186.358779] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 186.544715] Lustre: lustre-OST0001: new disk, initializing [ 186.548499] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 186.561467] Lustre: 9487:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 186.644541] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 192.159880] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 192.174013] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 192.250134] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 194.285266] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 207.094222] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 215.408773] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 221.610318] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing check_logdir /tmp/testlogs/ [ 228.030457] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing yml_node [ 232.408964] Lustre: DEBUG MARKER: Client: 2.17.57.87 [ 235.282598] Lustre: DEBUG MARKER: MDS: 2.17.57.87 [ 237.887508] Lustre: DEBUG MARKER: OSS: 2.17.57.87 [ 239.940551] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Aug 27 12:01:47 EDT 2026 [ 257.833299] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 259.280715] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 278.496101] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 278.504602] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 278.516082] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 279.526278] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 279.527689] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 281.787100] Lustre: server umount lustre-MDT0000 complete [ 285.630151] LustreError: 8425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787846555 with bad export cookie 1084752813630434918 [ 285.638604] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 285.647845] LustreError: 8425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 289.760484] LustreError: 6486:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 289.760597] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 289.779765] LustreError: 6486:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 6 previous similar messages [ 289.796479] Lustre: Skipped 2 previous similar messages [ 289.796895] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 289.819556] Lustre: Skipped 4 previous similar messages [ 290.787496] LustreError: 8420:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 290.804183] LustreError: 8420:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [ 292.104400] Lustre: server umount lustre-MDT0001 complete [ 295.994476] Lustre: server umount lustre-OST0000 complete [ 299.543729] Lustre: server umount lustre-OST0001 complete [ 305.912599] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_hostid [ 318.164594] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 357.842092] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 358.258045] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 358.305636] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 358.461388] Lustre: lustre-MDT0000: new disk, initializing [ 358.603108] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 358.643202] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 363.878628] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 376.326300] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 376.581103] Lustre: 15056:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 376.671975] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 376.678437] Lustre: Skipped 1 previous similar message [ 376.776327] Lustre: lustre-MDT0001: new disk, initializing [ 376.896917] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 376.940105] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 376.953239] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 382.768499] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 395.445219] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 400.610987] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 409.041695] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 409.482250] Lustre: lustre-OST0000: new disk, initializing [ 409.491524] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 409.505900] Lustre: 17025:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 409.635105] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 410.898762] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 410.911797] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 411.078423] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 416.327151] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 425.107608] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 429.574839] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 429.804186] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 434.276227] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 434.459885] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 436.716458] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 436.722332] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 436.730206] LustreError: Skipped 1 previous similar message [ 436.734415] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 436.739345] Lustre: Skipped 2 previous similar messages [ 436.743824] Lustre: Skipped 2 previous similar messages [ 441.834423] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 441.844868] Lustre: Skipped 2 previous similar messages [ 442.519603] Lustre: server umount lustre-OST0000 complete [ 446.944490] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 452.074770] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 452.081079] Lustre: Skipped 3 previous similar messages [ 452.670355] Lustre: server umount lustre-MDT0000 complete [ 456.235237] LustreError: 15048:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787846726 with bad export cookie 1084752813630436451 [ 456.240602] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 456.245419] LustreError: 15048:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 456.497129] Lustre: server umount lustre-MDT0001 complete [ 461.575843] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 12:05:28 (1787846728) [ 468.126612] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 468.733823] LustreError: 19289:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 468.894573] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 473.930864] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 474.081137] LustreError: 19290:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 478.193121] LustreError: 19289:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 483.876786] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 489.225139] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 498.567811] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 505.519467] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 514.087481] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 522.070636] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 531.296762] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 545.756689] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 553.449520] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 553.456810] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 553.483089] Lustre: Skipped 2 previous similar messages [ 553.509826] Lustre: Skipped 2 previous similar messages [ 558.859475] Lustre: server umount lustre-MDT0000 complete [ 563.334238] LustreError: 21844:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787846833 with bad export cookie 1084752813630437333 [ 563.340139] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 563.367630] LustreError: 21844:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 563.700051] LustreError: 19285:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 563.756132] LustreError: 19285:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [ 564.210915] Lustre: server umount lustre-MDT0001 complete [ 569.792247] Lustre: server umount lustre-OST0000 complete [ 595.374455] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 606.489298] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 606.939854] LustreError: 23694:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 606.954813] LustreError: 23694:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [ 607.041368] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 607.052162] Lustre: Skipped 2 previous similar messages [ 611.609867] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 620.351983] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 625.819293] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 629.125647] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 636.419441] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 643.856528] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 653.159616] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 653.301790] Lustre: lustre-OST0001: new disk, initializing [ 653.304908] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 653.312855] Lustre: 26009:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 655.252083] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 655.276154] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 655.442442] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 660.561138] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 670.053816] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 674.168716] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 687.589988] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 687.618206] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 687.628920] Lustre: Skipped 2 previous similar messages [ 687.634882] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 687.640792] Lustre: Skipped 3 previous similar messages [ 693.860383] Lustre: server umount lustre-MDT0000 complete [ 696.294651] LustreError: 23689:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 696.319826] LustreError: 23689:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 5 previous similar messages [ 697.406251] LustreError: 23676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787846967 with bad export cookie 1084752813630438180 [ 697.408063] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 697.420900] LustreError: 23676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 697.904577] Lustre: server umount lustre-MDT0001 complete [ 712.708801] Lustre: server umount lustre-OST0000 complete [ 726.398426] Lustre: server umount lustre-OST0001 complete [ 745.585851] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 758.011050] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 758.865523] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 758.880113] Lustre: Skipped 3 previous similar messages [ 763.491884] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 763.874131] LustreError: 29041:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 763.900090] LustreError: 29041:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [ 774.485447] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 779.148465] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 781.840112] Lustre: 30189:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 788.217840] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 795.169597] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 806.276600] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 813.661622] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 821.654976] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 825.048527] Lustre: 32066:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 831.458061] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 831.470843] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 831.502203] Lustre: Skipped 3 previous similar messages [ 831.507508] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 831.511670] Lustre: Skipped 4 previous similar messages [ 837.308769] Lustre: server umount lustre-MDT0000 complete [ 840.971794] LustreError: 29023:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787847110 with bad export cookie 1084752813630439475 [ 840.972897] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 840.981977] LustreError: 29023:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 841.355611] Lustre: server umount lustre-MDT0001 complete [ 856.224950] Lustre: server umount lustre-OST0000 complete [ 859.120729] Lustre: 3614:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787847112/real 1787847112] req@ffff9646820ade00 x1874692744187648/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787847128 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 861.151833] Lustre: 3612:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787847115/real 1787847115] req@ffff9646820af480 x1874692744187904/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787847131 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 862.428262] Lustre: server umount lustre-OST0001 complete [ 876.294937] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 12:12:23 (1787847143) [ 911.186258] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 921.647419] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 922.045441] LustreError: 34874:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 922.060477] LustreError: 34874:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 9 previous similar messages [ 926.699378] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 935.321115] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 940.386770] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 943.713161] Lustre: 36022:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 951.157494] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 957.979480] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 967.043275] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 973.405395] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 980.635957] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 984.609661] Lustre: 37904:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 992.225665] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 992.229791] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 992.241079] Lustre: Skipped 4 previous similar messages [ 992.254104] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 992.262822] Lustre: Skipped 4 previous similar messages [ 996.038716] Lustre: server umount lustre-MDT0000 complete [ 999.871932] LustreError: 38056:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787847269 with bad export cookie 1084752813630440581 [ 999.875452] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 999.882039] LustreError: 38056:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1000.177756] Lustre: server umount lustre-MDT0001 complete [ 1013.162596] Lustre: server umount lustre-OST0000 complete [ 1028.797347] Lustre: server umount lustre-OST0001 complete [ 1036.796502] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 12:15:04 (1787847304) [ 1054.152243] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 1063.784595] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1064.353606] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1064.357671] Lustre: Skipped 7 previous similar messages [ 1070.034633] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 1078.756532] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1083.069829] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 1085.898606] Lustre: 41194:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1093.276688] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1099.453234] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 1108.489553] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1114.800497] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 1121.613402] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1124.779936] Lustre: 43073:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1125.741247] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1125.746462] Lustre: Skipped 1 previous similar message [ 1130.477987] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1130.492145] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1130.521757] Lustre: Skipped 4 previous similar messages [ 1134.922976] Lustre: server umount lustre-MDT0000 complete [ 1138.281116] LustreError: 40025:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787847408 with bad export cookie 1084752813630441687 [ 1138.285602] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1138.296773] LustreError: 40025:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1144.659301] Lustre: server umount lustre-MDT0001 complete [ 1153.168641] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1163.232366] LustreError: 3610:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff96478b4de580 x1874692744286336/t0(0) o250->MGC192.168.206.150@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1168.441671] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 1173.997414] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1178.147033] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1183.733084] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1183.743696] Lustre: Skipped 1 previous similar message [ 1183.910775] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 1192.635894] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1194.503568] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1201.529703] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1203.685045] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1214.439666] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1215.817942] Lustre: server umount lustre-MDT0000 complete [ 1219.569828] LustreError: 43946:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1219.618362] LustreError: 43946:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 34 previous similar messages [ 1220.126061] LustreError: 40027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787847490 with bad export cookie 1084752813630442814 [ 1220.129940] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1220.143238] LustreError: 40027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1220.715628] Lustre: server umount lustre-MDT0001 complete [ 1235.114090] Lustre: server umount lustre-OST0000 complete [ 1248.480643] Lustre: server umount lustre-OST0001 complete [ 1257.844891] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 12:18:45 (1787847525) [ 1276.655617] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 1286.491618] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1292.393070] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 1301.561798] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1306.960727] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 1309.831124] Lustre: 48835:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1316.158623] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1322.709960] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 1331.193609] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1337.634799] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 1346.013270] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1351.324940] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1368.544312] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1368.554652] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1368.575298] Lustre: Skipped 21 previous similar messages [ 1381.343142] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1381.677884] Lustre: server umount lustre-MDT0000 complete [ 1385.249377] LustreError: 47665:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787847655 with bad export cookie 1084752813630443598 [ 1385.255111] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1385.267862] LustreError: 47665:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1385.579283] Lustre: server umount lustre-MDT0001 complete [ 1399.537959] Lustre: server umount lustre-OST0000 complete [ 1413.734348] Lustre: server umount lustre-OST0001 complete [ 1426.035592] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 12:21:33 (1787847693) [ 1429.158360] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1430.821508] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 12:21:38 (1787847698) [ 1441.654510] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_hostid [ 1450.452409] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 1473.261750] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1473.525983] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1473.622390] Lustre: lustre-MDT0000: new disk, initializing [ 1473.742478] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1477.964306] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 1486.718188] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1493.817566] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1494.220619] Lustre: lustre-OST0000: new disk, initializing [ 1494.230296] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1494.244487] Lustre: Skipped 1 previous similar message [ 1494.259860] Lustre: 54913:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1496.204885] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1496.222502] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1496.261525] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1499.925541] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 1510.440753] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1513.006737] Lustre: DEBUG MARKER: create test files [ 1547.241448] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1547.248321] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1547.265099] Lustre: Skipped 12 previous similar messages [ 1550.558716] Lustre: server umount lustre-OST0000 complete [ 1555.080432] Lustre: server umount lustre-MDT0000 complete [ 1566.308659] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [ 1568.982668] Key type lgssc unregistered [ 1569.287801] LNet: 56551:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1569.294638] LNetError: 56551:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1569.313739] LNet: Removed LNI 192.168.206.150@tcp [ 1570.449147] Key type .llcrypt unregistered [ 1570.460076] Key type ._llcrypt unregistered [ 1597.799246] Key type ._llcrypt registered [ 1597.801395] Key type .llcrypt registered [ 1597.905628] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 1599.175572] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1599.234045] alg: No test for adler32 (adler32-zlib) [ 1600.318833] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [ 1600.613594] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [ 1602.343274] Key type lgssc registered [ 1603.907876] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1611.403328] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1611.421538] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1612.884180] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1616.833300] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 1624.464458] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1631.228783] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1631.683864] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1637.498937] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 1637.870683] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1637.960394] Lustre: 59289:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1637.970351] Lustre: 59289:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1637.978973] Lustre: 59289:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1637.985691] Lustre: 59289:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1637.990612] Lustre: 59289:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1637.995866] Lustre: 59289:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1644.711394] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1650.598133] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1655.976726] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1663.458741] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1663.482260] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1668.576839] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1669.484296] Lustre: server umount lustre-OST0000 complete [ 1673.846319] Lustre: server umount lustre-MDT0000 complete [ 1685.485587] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [ 1688.215114] Key type lgssc unregistered [ 1688.580862] LNet: 60901:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1688.586963] LNetError: 60901:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1688.613989] LNet: Removed LNI 192.168.206.150@tcp [ 1689.624190] Key type .llcrypt unregistered [ 1689.626817] Key type ._llcrypt unregistered [ 1725.638580] Key type ._llcrypt registered [ 1725.640666] Key type .llcrypt registered [ 1725.737828] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 1727.001203] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1727.044964] alg: No test for adler32 (adler32-zlib) [ 1728.190900] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [ 1728.492804] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [ 1730.247155] Key type lgssc registered [ 1731.681293] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1740.181656] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1740.220477] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1741.701413] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1745.897483] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 1753.481375] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1761.002043] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1761.414718] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1766.534151] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1766.712827] Lustre: 63739:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1766.729415] Lustre: 63739:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1766.736833] Lustre: 63739:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1766.741647] Lustre: 63739:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1766.751818] Lustre: 63739:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1766.764815] Lustre: 63739:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1769.391805] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 1777.632700] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1783.417731] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1797.601744] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1797.601748] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1797.612371] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1799.120633] Lustre: server umount lustre-OST0000 complete [ 1804.970355] Lustre: server umount lustre-MDT0000 complete [ 1817.387833] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [ 1820.361277] Key type lgssc unregistered [ 1820.774567] LNet: 65300:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1820.789553] LNetError: 65300:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1820.814171] LNet: Removed LNI 192.168.206.150@tcp [ 1821.658116] Key type .llcrypt unregistered [ 1821.659818] Key type ._llcrypt unregistered [ 1842.954891] Key type ._llcrypt registered [ 1842.957294] Key type .llcrypt registered [ 1843.045710] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_hostid [ 1859.478937] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 1860.557523] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1860.685976] alg: No test for adler32 (adler32-zlib) [ 1861.776883] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [ 1861.994850] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [ 1863.663196] Key type lgssc registered [ 1865.083796] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1904.814895] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1904.872205] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1906.257460] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1906.293173] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1906.396625] Lustre: lustre-MDT0000: new disk, initializing [ 1906.554562] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1906.590108] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1911.469330] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 1923.667567] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1923.813682] Lustre: 68452:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1923.926226] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1923.936264] Lustre: Skipped 1 previous similar message [ 1924.058329] Lustre: lustre-MDT0001: new disk, initializing [ 1924.232314] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1924.335517] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1924.357364] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1929.280633] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 1939.885462] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1946.972609] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1954.018287] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1954.486626] Lustre: lustre-OST0000: new disk, initializing [ 1954.500586] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1954.531087] Lustre: 70426:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1954.717606] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1961.238816] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 1964.157937] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1964.162886] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1964.315881] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1971.575797] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1976.063621] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1976.257535] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1981.423634] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1981.665702] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1984.995684] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1984.998163] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1984.999167] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1985.015125] LustreError: Skipped 1 previous similar message [ 1985.033610] Lustre: Skipped 1 previous similar message [ 1990.113479] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1990.131904] Lustre: Skipped 2 previous similar messages [ 1995.240084] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1995.248856] Lustre: Skipped 1 previous similar message [ 1998.303129] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1998.473566] Lustre: server umount lustre-OST0000 complete [ 2005.489021] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2005.491662] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2005.508619] Lustre: Skipped 1 previous similar message [ 2005.519711] Lustre: Skipped 1 previous similar message [ 2008.681349] Lustre: server umount lustre-MDT0000 complete [ 2012.647584] LustreError: 68443:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787848282 with bad export cookie 4515763571541559939 [ 2012.650225] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2012.666349] LustreError: 68443:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2013.274119] Lustre: server umount lustre-MDT0001 complete [ 2022.484543] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 12:31:29 (1787848289) [ 2033.246218] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 2073.995392] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_hostid [ 2084.284328] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 2121.612194] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2121.827505] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2121.869260] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2121.969906] Lustre: lustre-MDT0000: new disk, initializing [ 2122.055546] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2122.082214] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2127.135509] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 2138.412927] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2138.545675] Lustre: 76447:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2138.597419] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2138.602545] Lustre: Skipped 1 previous similar message [ 2138.608374] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 2138.694471] Lustre: lustre-MDT0001: new disk, initializing [ 2138.804864] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2138.841170] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2138.851920] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2144.235375] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 2153.701893] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2159.427584] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2166.647166] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2166.890707] Lustre: lustre-OST0000: new disk, initializing [ 2166.895563] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2166.900315] Lustre: 78413:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2166.989276] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2173.943589] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 2175.056969] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2175.066298] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2175.226817] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2184.177438] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2189.273640] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2189.541932] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2194.501237] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2194.724920] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2201.064938] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2201.067268] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2201.072975] Lustre: Skipped 1 previous similar message [ 2201.079871] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2201.089941] LustreError: Skipped 1 previous similar message [ 2201.106696] Lustre: Skipped 1 previous similar message [ 2202.781660] Lustre: server umount lustre-OST0000 complete [ 2210.785216] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2210.790964] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2212.846829] Lustre: server umount lustre-MDT0000 complete [ 2215.969818] LustreError: 76440:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787848485 with bad export cookie 4515763571541560821 [ 2215.979426] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2216.242971] Lustre: server umount lustre-MDT0001 complete [ 2226.492978] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 12:34:53 (1787848493) [ 2235.023138] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2235.360344] LustreError: 80828:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2235.378141] LustreError: 80828:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 3 previous similar messages [ 2235.418932] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2239.873364] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 2240.491680] LustreError: 80829:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2245.600323] LustreError: 80828:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2248.608455] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2252.979155] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 2260.765743] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2266.245690] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2272.778167] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2273.094638] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2273.101639] Lustre: Skipped 1 previous similar message [ 2278.809278] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 2286.293561] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2289.608794] Lustre: 83500:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 3ba11634-4783-43dd-85c5-fa1373b02e14 at adminstrative request [ 2290.668515] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2290.686427] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2300.898670] Lustre: MGS: Client 3ba11634-4783-43dd-85c5-fa1373b02e14 (at 0@lo) reconnecting [ 2300.905098] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x3eab370af3625167 to 0x3eab370af3625407 [ 2300.921603] Lustre: MGC192.168.206.150@tcp: Connection restored to 0@lo (at 0@lo) [ 2327.180915] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2327.411960] Lustre: lustre-OST0001: new disk, initializing [ 2327.418497] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2327.424864] Lustre: 83898:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2333.934193] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 2336.375171] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2336.388581] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2336.486811] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2343.327854] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2346.978199] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2346.979112] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2346.989191] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2346.989197] Lustre: Skipped 3 previous similar messages [ 2347.015248] Lustre: Skipped 2 previous similar messages [ 2352.496390] Lustre: server umount lustre-OST0001 complete [ 2357.233702] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2357.236171] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2357.241252] LustreError: Skipped 1 previous similar message [ 2357.243946] LustreError: 82650:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2357.256310] Lustre: Skipped 1 previous similar message [ 2357.270958] LustreError: 82650:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [ 2362.357408] LustreError: 82734:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2362.387079] LustreError: 82734:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [ 2363.302389] Lustre: server umount lustre-OST0000 complete [ 2366.944344] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2381.279785] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2381.463211] Lustre: server umount lustre-MDT0000 complete [ 2382.816332] LustreError: 80828:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2382.838840] LustreError: 80828:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [ 2385.167833] LustreError: 80810:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787848655 with bad export cookie 4515763571541562375 [ 2385.173418] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2385.179153] LustreError: 80810:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2385.412325] Lustre: server umount lustre-MDT0001 complete [ 2393.373550] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 12:37:40 (1787848660) [ 2395.038385] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2396.761973] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 12:37:44 (1787848664) [ 2398.502331] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2400.328362] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 12:37:47 (1787848667) [ 2401.777664] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2403.880308] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 12:37:51 (1787848671) [ 2405.399979] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2407.388405] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 12:37:54 (1787848674) [ 2408.978884] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2415.334748] loop: module loaded [ 2415.362082] loop0: detected capacity change from 0 to 4294967296 [ 2415.411713] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2417.184514] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2420.510627] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2424.719240] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2451.820295] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2451.856744] LustreError: 88023:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2451.861325] LustreError: 88023:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2451.868126] LustreError: 88023:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2451.883255] LustreError: 88023:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2471.726620] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_hostid [ 2483.406366] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 2525.425209] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2525.677584] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2525.721664] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2525.840257] Lustre: lustre-MDT0000: new disk, initializing [ 2525.945759] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2525.952038] Lustre: Skipped 1 previous similar message [ 2525.965075] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2531.319491] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 2542.099355] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2542.218426] Lustre: 90888:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2542.508439] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2547.398462] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 2557.006390] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2562.436096] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2569.229888] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2569.595230] Lustre: 92859:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2577.572246] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2577.575851] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 2577.727699] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2589.315543] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2595.181887] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2595.431589] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2600.399680] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2600.651384] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2603.488540] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2603.494285] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2603.512360] Lustre: Skipped 3 previous similar messages [ 2603.533277] Lustre: Skipped 15 previous similar messages [ 2617.311572] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2617.552237] Lustre: server umount lustre-OST0000 complete [ 2627.940989] Lustre: server umount lustre-MDT0000 complete [ 2629.089769] LustreError: 90901:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2629.119877] LustreError: 90901:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [ 2632.168538] LustreError: 91685:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787848902 with bad export cookie 4515763571541562991 [ 2632.171198] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2632.177116] LustreError: 91685:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 2632.393953] Lustre: server umount lustre-MDT0001 complete [ 2643.261953] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 12:41:49 (1787848909) [ 2651.000461] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2655.844892] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 2661.860720] LustreError: 95273:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2661.871426] LustreError: 95273:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 2 previous similar messages [ 2664.600024] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2664.955818] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2664.962756] Lustre: Skipped 3 previous similar messages [ 2669.250396] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 2678.544769] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2684.567752] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2691.864928] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2698.048560] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 2705.424385] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2710.478628] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2710.716713] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2715.731042] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2716.098855] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2723.736244] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2723.939791] Lustre: lustre-OST0001: new disk, initializing [ 2723.943139] Lustre: Skipped 2 previous similar messages [ 2723.946333] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2723.949781] Lustre: Skipped 3 previous similar messages [ 2723.954517] Lustre: 98643:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2730.605781] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 2733.147792] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2733.157116] Lustre: Skipped 2 previous similar messages [ 2733.162627] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2733.281832] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2740.240472] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2745.263342] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2745.540090] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2750.338277] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2750.508571] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2770.102365] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2794.202394] Lustre: Failing over lustre-MDT0000 [ 2794.603549] Lustre: server umount lustre-MDT0000 complete [ 2794.993471] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2794.996240] LustreError: 95269:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2795.007512] Lustre: Skipped 4 previous similar messages [ 2795.027581] LustreError: 95269:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 3 previous similar messages [ 2816.480378] Lustre: 66677:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787849070/real 1787849070] req@0000000045e13744 x1874694642087808/t0(0) o400->MGC192.168.206.150@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787849086 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2816.521249] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2819.938414] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2819.941278] LDISKFS-fs (dm-0): recovery complete [ 2819.947573] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2827.269231] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2829.856994] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2829.857638] Lustre: 95270:0:(ldlm_lib.c:2120:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2832.357186] Lustre: 95273:0:(ldlm_lib.c:2120:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2832.382651] Lustre: 95273:0:(ldlm_lib.c:2120:extend_recovery_timer()) Skipped 1 previous similar message [ 2832.411546] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2832.513695] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2832.839123] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2833.170348] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2833.187152] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) Skipped 1 previous similar message [ 2833.218497] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 2833.543133] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2833.554082] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) Skipped 1 previous similar message [ 2833.574803] Lustre: 101496:0:(ldlm_lib.c:2120:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2833.598935] Lustre: 101496:0:(ldlm_lib.c:2120:extend_recovery_timer()) Skipped 7 previous similar messages [ 2834.251294] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2834.276573] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) Skipped 2 previous similar messages [ 2834.603027] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2834.614181] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) Skipped 2 previous similar messages [ 2835.601133] Lustre: 101496:0:(ldlm_lib.c:2120:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2835.619149] Lustre: 101496:0:(ldlm_lib.c:2120:extend_recovery_timer()) Skipped 5 previous similar messages [ 2836.280578] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2836.309759] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) Skipped 5 previous similar messages [ 2836.639781] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2836.647493] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) Skipped 5 previous similar messages [ 2839.877088] Lustre: 101496:0:(ldlm_lib.c:2120:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2839.892414] Lustre: 101496:0:(ldlm_lib.c:2120:extend_recovery_timer()) Skipped 12 previous similar messages [ 2840.534874] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2840.549947] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) Skipped 12 previous similar messages [ 2840.871285] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2840.884683] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) Skipped 12 previous similar messages [ 2847.896367] Lustre: 101496:0:(ldlm_lib.c:2120:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2847.917994] Lustre: 101496:0:(ldlm_lib.c:2120:extend_recovery_timer()) Skipped 24 previous similar messages [ 2848.560952] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2848.582235] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) Skipped 24 previous similar messages [ 2848.912349] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2848.928949] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) Skipped 24 previous similar messages [ 2863.896046] Lustre: 101496:0:(ldlm_lib.c:2120:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2863.921885] Lustre: 101496:0:(ldlm_lib.c:2120:extend_recovery_timer()) Skipped 49 previous similar messages [ 2864.568233] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2864.597293] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) Skipped 49 previous similar messages [ 2864.919479] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2864.926243] LustreError: 101496:0:(ldlm_lib.c:2733:replay_request_or_update()) Skipped 49 previous similar messages [ 2889.072089] Lustre: 101496:0:(ldlm_lib.c:2429:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2889.087904] Lustre: 101496:0:(ldlm_lib.c:2439:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2889.104595] Lustre: 101496:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 13d87868-5fd4-4ff0-b898-286ec8478959@192.168.206.50@tcp [ 2889.132906] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2889.146303] Lustre: 101496:0:(ldlm_lib.c:1949:abort_req_replay_queue()) @@@ aborted: req@00000000bd197b72 x1874694624055680/t0(8589934773) o101->13d87868-5fd4-4ff0-b898-286ec8478959@192.168.206.50@tcp:109/0 lens 592/0 e 0 to 0 dl 1787849169 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2889.193869] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2889.209649] Lustre: lustre-MDT0000: Denying connection for new client 13d87868-5fd4-4ff0-b898-286ec8478959 (at 192.168.206.50@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2889.226825] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2889.265466] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2889.276857] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2889.278786] Lustre: Skipped 2 previous similar messages [ 2889.324746] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 2889.325969] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2909.156701] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2909.161811] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2909.177017] Lustre: Skipped 2 previous similar messages [ 2909.190543] Lustre: Skipped 7 previous similar messages [ 2913.402561] Lustre: server umount lustre-OST0000 complete [ 2917.859711] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2923.290633] Lustre: server umount lustre-OST0001 complete [ 2934.157893] Lustre: server umount lustre-MDT0000 complete [ 2934.753255] LustreError: 95273:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2934.792610] LustreError: 95273:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 45 previous similar messages [ 2938.643367] LustreError: 95253:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787849208 with bad export cookie 4515763571541614210 [ 2938.656701] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2938.665246] LustreError: 95253:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2939.088708] Lustre: server umount lustre-MDT0001 complete [ 2949.251366] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 12:46:56 (1787849216) [ 2956.821072] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2956.829059] Lustre: *** cfs_fail_loc=197, val=0*** [ 2972.319772] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2977.439959] LustreError: 103469:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.150@tcp: failed processing log, type 4: rc = -110 [ 2982.595658] Lustre: *** cfs_fail_loc=197, val=0*** [ 2982.604161] Lustre: Skipped 65 previous similar messages [ 3003.168494] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3003.176724] Lustre: Skipped 3 previous similar messages [ 3010.290722] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 3018.272941] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3020.523407] Lustre: server umount lustre-OST0000 complete [ 3029.374701] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 12:48:16 (1787849296) [ 3049.741623] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3054.363091] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 3063.110714] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3066.772116] LustreError: 105123:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3066.781538] LustreError: 105123:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3066.788224] LustreError: 105123:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 16, retries 0, failed: rc = -5 [ 3067.227197] Lustre: server umount lustre-MDT0000 complete [ 3099.283734] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_hostid [ 3112.107615] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 3147.243502] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3147.523085] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3147.593654] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3147.708518] Lustre: lustre-MDT0000: new disk, initializing [ 3147.792863] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3152.394254] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 3162.672289] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3162.824180] Lustre: 109297:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3163.181096] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3167.557329] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 3178.042994] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3184.561764] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3192.395936] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3192.833797] Lustre: 111267:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3199.181204] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3199.999523] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 3210.139785] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3215.106402] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3215.380304] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3220.711727] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3220.910506] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3225.057175] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3225.073826] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3225.079311] LustreError: Skipped 1 previous similar message [ 3225.084206] Lustre: Skipped 4 previous similar messages [ 3225.092379] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3225.104995] Lustre: Skipped 5 previous similar messages [ 3237.343489] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3237.631767] Lustre: server umount lustre-OST0000 complete [ 3250.041486] Lustre: server umount lustre-MDT0000 complete [ 3250.662430] LustreError: 109304:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3250.690680] LustreError: 109304:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 5 previous similar messages [ 3256.485316] LustreError: 109292:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787849526 with bad export cookie 4515763571541624822 [ 3256.503848] LustreError: 109292:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3256.506158] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3257.062265] Lustre: server umount lustre-MDT0001 complete [ 3267.311250] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 12:52:14 (1787849534) [ 3277.772660] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [ 3281.306523] Key type lgssc unregistered [ 3281.654663] LNet: 113760:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3281.671415] LNetError: 113760:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3281.708252] LNet: Removed LNI 192.168.206.150@tcp [ 3282.728412] Key type .llcrypt unregistered [ 3282.731222] Key type ._llcrypt unregistered [ 3302.864467] Key type ._llcrypt registered [ 3302.867187] Key type .llcrypt registered [ 3302.951220] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_hostid [ 3317.442450] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 3318.732663] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3318.798728] alg: No test for adler32 (adler32-zlib) [ 3319.897529] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [ 3320.181501] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [ 3321.895536] Key type lgssc registered [ 3322.999783] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3363.586428] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3363.632043] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3364.891748] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3364.950562] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3365.024321] Lustre: lustre-MDT0000: new disk, initializing [ 3365.108463] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3365.123209] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3368.997595] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 3379.748771] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3379.976252] Lustre: lustre-OST0000: new disk, initializing [ 3379.980152] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3379.984566] Lustre: Skipped 1 previous similar message [ 3379.989733] Lustre: 117936:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3380.059351] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3385.653490] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 3388.511434] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3388.531741] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3388.554443] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3395.689664] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3395.908128] Lustre: lustre-OST0001: new disk, initializing [ 3395.935108] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3395.942483] Lustre: 118799:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3396.041028] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3401.859951] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3401.877716] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3401.913634] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3402.152408] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 3411.946759] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3411.952973] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3411.956733] Lustre: Skipped 1 previous similar message [ 3411.967935] Lustre: Skipped 1 previous similar message [ 3416.768548] Lustre: server umount lustre-MDT0000 complete [ 3423.283258] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3427.958849] LustreError: 118798:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787849697 with bad export cookie 8850695437145507817 [ 3427.963278] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3427.971969] LustreError: 118798:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3428.034623] Lustre: server umount lustre-OST0000 complete [ 3431.981135] Lustre: server umount lustre-OST0001 complete [ 3440.387797] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 12:55:07 (1787849707) [ 3461.611514] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 12:55:28 (1787849728) [ 3472.443989] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 3497.993975] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 12:56:04 (1787849764) [ 3536.318104] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_hostid [ 3546.656392] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 3581.699264] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3581.935749] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3581.970902] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3582.029595] Lustre: lustre-MDT0000: new disk, initializing [ 3582.086995] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3582.099905] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3586.231393] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 3595.181557] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3595.283422] Lustre: 126551:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3595.330255] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3595.334485] Lustre: Skipped 1 previous similar message [ 3595.398165] Lustre: lustre-MDT0001: new disk, initializing [ 3595.496235] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3595.525555] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3595.536867] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3599.212091] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 3608.015285] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3613.204794] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3619.069056] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3619.292462] Lustre: lustre-OST0000: new disk, initializing [ 3619.295820] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3619.303899] Lustre: 128518:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3619.376358] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3624.807109] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 3626.523698] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3626.529423] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3626.608696] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3633.588209] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3638.010895] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3638.248321] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3643.020742] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3643.363555] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3646.950016] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3646.952663] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3646.963264] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3646.976030] Lustre: Skipped 1 previous similar message [ 3651.536500] Lustre: server umount lustre-OST0000 complete [ 3657.183821] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3657.186489] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3657.209421] Lustre: Skipped 1 previous similar message [ 3657.221918] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3657.228770] Lustre: Skipped 1 previous similar message [ 3661.572128] Lustre: server umount lustre-MDT0000 complete [ 3666.081198] LustreError: 126544:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787849935 with bad export cookie 8850695437145509035 [ 3666.082958] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3666.092018] LustreError: 126544:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3666.458685] Lustre: server umount lustre-MDT0001 complete [ 3675.343556] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 12:59:02 (1787849942) [ 3682.519979] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3682.955912] LustreError: 130931:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3682.999340] LustreError: 130931:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 3 previous similar messages [ 3683.034646] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3686.948173] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 3688.428378] LustreError: 130931:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3688.447250] LustreError: 130931:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [ 3692.519873] LustreError: 130932:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3695.280024] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3699.901570] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 3707.262241] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3713.249148] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3720.412720] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3720.702809] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3720.711178] Lustre: Skipped 1 previous similar message [ 3727.791404] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 3736.930248] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3745.027385] Lustre: *** cfs_fail_loc=159, val=0*** [ 3761.253308] Lustre: lustre-MDT0000: Client eb2050cb-85d1-4e6d-9163-16a92df71cff (at 192.168.206.50@tcp) reconnecting [ 3762.590068] Lustre: *** cfs_fail_loc=159, val=0*** [ 3762.591936] Lustre: Skipped 5 previous similar messages [ 3779.184551] Lustre: lustre-MDT0000: Client eb2050cb-85d1-4e6d-9163-16a92df71cff (at 192.168.206.50@tcp) reconnecting [ 3784.167891] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3784.183207] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3784.195682] Lustre: Skipped 1 previous similar message [ 3784.211330] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3786.209268] LustreError: 132757:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3786.211111] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3786.341444] Lustre: server umount lustre-OST0000 complete [ 3791.350482] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3791.402819] Lustre: Skipped 1 previous similar message [ 3791.427866] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3791.438167] Lustre: Skipped 1 previous similar message [ 3796.600264] Lustre: server umount lustre-MDT0000 complete [ 3797.484086] LustreError: 130932:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3797.512140] LustreError: 130932:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [ 3800.868768] LustreError: 130912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787850070 with bad export cookie 8850695437145509924 [ 3800.872868] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3800.883179] LustreError: 130912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3801.240310] Lustre: server umount lustre-MDT0001 complete [ 3814.238847] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [ 3816.890765] Key type lgssc unregistered [ 3817.273033] LNet: 134729:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3817.280744] LNetError: 134729:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3817.303956] LNet: Removed LNI 192.168.206.150@tcp [ 3818.230365] Key type .llcrypt unregistered [ 3818.234435] Key type ._llcrypt unregistered [ 3840.986354] Key type ._llcrypt registered [ 3840.991281] Key type .llcrypt registered [ 3841.071335] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 3841.813770] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3841.942032] alg: No test for adler32 (adler32-zlib) [ 3842.919935] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [ 3843.179223] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [ 3844.823193] Key type lgssc registered [ 3846.028834] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3853.484788] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3853.517462] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3855.020819] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3859.458028] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 3864.545402] LustreError: 136142:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3864.584318] LustreError: 136142:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 2 previous similar messages [ 3868.213861] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3868.601280] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3873.064289] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 3880.502280] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3886.840218] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3894.074135] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3894.455217] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3899.572222] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3900.863689] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 3907.891795] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3921.093430] Lustre: *** cfs_fail_loc=159, val=0*** [ 3936.345040] Lustre: lustre-MDT0000: Client 1af3a153-ffe0-4a98-b807-170d50c8383d (at 192.168.206.50@tcp) reconnecting [ 3938.883018] Lustre: *** cfs_fail_loc=159, val=0*** [ 3938.894687] Lustre: Skipped 1 previous similar message [ 3954.297868] Lustre: lustre-MDT0001: Client 1af3a153-ffe0-4a98-b807-170d50c8383d (at 192.168.206.50@tcp) reconnecting [ 3956.256502] Lustre: *** cfs_fail_loc=159, val=0*** [ 3956.258580] Lustre: Skipped 2 previous similar messages [ 3972.723457] Lustre: lustre-MDT0001: Client 1af3a153-ffe0-4a98-b807-170d50c8383d (at 192.168.206.50@tcp) reconnecting [ 3988.254229] Lustre: 139211:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3991.171907] Lustre: *** cfs_fail_loc=159, val=0*** [ 3991.181472] Lustre: Skipped 1 previous similar message [ 4006.499609] Lustre: lustre-MDT0000: Client 54dcc6fc-d786-4c52-aa02-7fbbd8914929 (at 192.168.206.50@tcp) reconnecting [ 4008.271591] Lustre: *** cfs_fail_loc=159, val=0*** [ 4008.284970] Lustre: Skipped 16 previous similar messages [ 4024.939647] Lustre: lustre-MDT0000: Client 54dcc6fc-d786-4c52-aa02-7fbbd8914929 (at 192.168.206.50@tcp) reconnecting [ 4026.788201] Lustre: 139502:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4032.992178] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4033.006659] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4033.028140] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4033.028609] Lustre: Skipped 1 previous similar message [ 4033.043210] Lustre: Skipped 1 previous similar message [ 4037.100968] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4038.457318] Lustre: server umount lustre-OST0000 complete [ 4042.209500] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4042.230324] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4042.236413] Lustre: Skipped 2 previous similar messages [ 4047.332328] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4047.336767] Lustre: Skipped 1 previous similar message [ 4048.170861] Lustre: server umount lustre-MDT0000 complete [ 4051.027794] LustreError: 139332:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787850320 with bad export cookie 13708525340336199157 [ 4051.034778] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4051.040791] LustreError: 139332:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4051.346126] Lustre: server umount lustre-MDT0001 complete [ 4060.971340] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [ 4064.056091] Key type lgssc unregistered [ 4064.327086] LNet: 140559:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4064.336707] LNetError: 140559:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4065.385906] LNet: Removed LNI 192.168.206.150@tcp [ 4066.214839] Key type .llcrypt unregistered [ 4066.218943] Key type ._llcrypt unregistered [ 4092.956819] Key type ._llcrypt registered [ 4092.958368] Key type .llcrypt registered [ 4093.090046] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 4094.359243] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4094.384446] alg: No test for adler32 (adler32-zlib) [ 4095.543745] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [ 4095.750104] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [ 4097.415278] Key type lgssc registered [ 4098.505927] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4106.295557] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4106.308498] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4107.714092] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4111.708514] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 4117.992094] LustreError: 141975:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4118.008231] LustreError: 141975:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 2 previous similar messages [ 4119.149496] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4119.454663] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4123.718179] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 4130.102986] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4135.573599] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4142.392714] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4142.643040] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4144.693400] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4148.112429] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 4152.891930] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4155.140551] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4161.607816] Lustre: 144557:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4164.422864] Lustre: 144605:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4165.837595] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4166.685727] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4169.965617] Lustre: 144754:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4170.707439] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4171.432038] Lustre: 144852:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4175.328186] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4175.337117] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4175.347381] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4178.402295] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4178.403421] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4178.420578] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4178.423870] Lustre: Skipped 1 previous similar message [ 4181.213696] Lustre: server umount lustre-OST0000 complete [ 4186.080450] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4186.089081] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4186.102775] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4188.640419] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4188.641625] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4193.760555] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4193.768036] Lustre: Skipped 2 previous similar messages [ 4198.367871] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4198.600927] Lustre: server umount lustre-MDT0000 complete [ 4198.882153] LustreError: 141971:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4198.893857] LustreError: 141971:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [ 4202.148620] LustreError: 141956:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787850472 with bad export cookie 11170662257936546830 [ 4202.163629] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4202.479865] Lustre: server umount lustre-MDT0001 complete [ 4211.806741] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [ 4214.794350] Key type lgssc unregistered [ 4215.149573] LNet: 145917:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4215.163490] LNetError: 145917:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4215.182807] LNet: Removed LNI 192.168.206.150@tcp [ 4216.029955] Key type .llcrypt unregistered [ 4216.033051] Key type ._llcrypt unregistered [ 4238.616121] Key type ._llcrypt registered [ 4238.618503] Key type .llcrypt registered [ 4238.718954] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 4239.992269] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4240.123124] alg: No test for adler32 (adler32-zlib) [ 4241.196601] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [ 4241.538690] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [ 4243.191108] Key type lgssc registered [ 4244.143809] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4251.256059] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4251.282376] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4252.737113] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4256.941610] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 4262.880733] LustreError: 147332:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4262.892292] LustreError: 147332:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 2 previous similar messages [ 4266.439524] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4266.985912] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4271.143119] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 4279.363799] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4285.950742] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4293.220151] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4293.774186] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4297.919796] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4301.164355] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 4309.274973] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4311.220432] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4317.726040] Lustre: *** cfs_fail_loc=159, val=0*** [ 4334.165067] Lustre: lustre-MDT0000: Client 574bc80e-dbc2-40ad-8b4d-d1ff6008fa3e (at 192.168.206.50@tcp) reconnecting [ 4340.297753] Lustre: server umount lustre-OST0000 complete [ 4347.361512] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4347.363349] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4347.384191] Lustre: Skipped 1 previous similar message [ 4352.484147] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4352.490879] Lustre: Skipped 1 previous similar message [ 4354.630872] Lustre: server umount lustre-MDT0000 complete [ 4357.607443] LustreError: 147331:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4358.482583] LustreError: 148072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787850628 with bad export cookie 7158118201263846483 [ 4358.484732] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4358.499813] LustreError: 148072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4369.096871] Lustre: server umount lustre-MDT0001 complete [ 4380.880505] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [ 4383.803739] Key type lgssc unregistered [ 4384.067076] LNet: 151002:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4384.075277] LNetError: 151002:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4384.088082] LNet: Removed LNI 192.168.206.150@tcp [ 4384.912984] Key type .llcrypt unregistered [ 4384.916271] Key type ._llcrypt unregistered [ 4409.264346] Key type ._llcrypt registered [ 4409.267339] Key type .llcrypt registered [ 4409.374721] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 4410.609705] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4410.663867] alg: No test for adler32 (adler32-zlib) [ 4411.694743] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [ 4411.928881] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [ 4413.664584] Key type lgssc registered [ 4415.069968] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4423.162039] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4423.174833] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4424.530614] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4428.593793] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 4434.916544] LustreError: 152414:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4434.947778] LustreError: 152414:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 2 previous similar messages [ 4436.938242] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4437.336086] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4441.813968] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 4450.194961] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4456.566795] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4463.485124] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4463.840440] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4469.992152] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4470.260924] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 4477.699257] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4482.296881] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4502.644279] Lustre: lustre-MDT0001: evicting 2f297bbf-859b-4368-8c5c-b69effb6413e (at 192.168.206.50@tcp) by administrative request [ 4502.656253] Lustre: Skipped 1 previous similar message [ 4531.682793] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4531.693084] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4531.705979] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4533.730628] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4533.732762] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4536.825820] Lustre: server umount lustre-OST0000 complete [ 4543.979807] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4543.999230] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4544.014745] Lustre: Skipped 1 previous similar message [ 4544.026028] Lustre: Skipped 2 previous similar messages [ 4546.547245] Lustre: server umount lustre-MDT0000 complete [ 4549.090209] LustreError: 152409:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4549.121448] LustreError: 152409:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [ 4550.673774] LustreError: 152395:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787850820 with bad export cookie 7773210159311029598 [ 4550.679306] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4550.685025] LustreError: 152395:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4551.105512] Lustre: server umount lustre-MDT0001 complete [ 4563.941456] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [ 4568.991145] Key type lgssc unregistered [ 4569.342511] LNet: 156500:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4569.349555] LNetError: 156500:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4569.366405] LNet: Removed LNI 192.168.206.150@tcp [ 4571.360899] Key type .llcrypt unregistered [ 4571.362523] Key type ._llcrypt unregistered [ 4654.110620] Key type ._llcrypt registered [ 4654.113826] Key type .llcrypt registered [ 4654.241147] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 4655.185407] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4655.198112] alg: No test for adler32 (adler32-zlib) [ 4656.257475] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [ 4656.616791] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [ 4658.327139] Key type lgssc registered [ 4659.502826] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4667.442302] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4667.461110] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4668.900137] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4673.151342] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 4678.114395] LustreError: 158990:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4678.124487] LustreError: 158990:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 2 previous similar messages [ 4681.395555] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4681.760910] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4686.036621] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 4693.511625] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4698.989602] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4705.944989] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4706.223454] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 4706.354399] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4711.413954] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4711.687453] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 4713.472095] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4718.334256] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4726.754216] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4726.762229] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4726.773152] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4728.803323] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4728.811667] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4728.824754] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4731.413277] Lustre: server umount lustre-OST0000 complete [ 4738.017905] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4738.024764] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4738.033987] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4738.042401] Lustre: Skipped 1 previous similar message [ 4740.902191] Lustre: server umount lustre-MDT0000 complete [ 4743.834573] LustreError: 158971:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787851013 with bad export cookie 4172729395000777450 [ 4743.840665] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4743.845226] LustreError: 158971:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4744.131250] Lustre: server umount lustre-MDT0001 complete [ 4753.037344] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [ 4755.282616] Key type lgssc unregistered [ 4755.596092] LNet: 162533:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4755.605315] LNetError: 162533:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4755.625297] LNet: Removed LNI 192.168.206.150@tcp [ 4756.522221] Key type .llcrypt unregistered [ 4756.528045] Key type ._llcrypt unregistered [ 4778.214391] Key type ._llcrypt registered [ 4778.216609] Key type .llcrypt registered [ 4778.296381] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 4779.434797] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4779.454246] alg: No test for adler32 (adler32-zlib) [ 4780.553989] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [ 4780.782948] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [ 4782.519607] Key type lgssc registered [ 4783.814619] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4798.367396] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 13:17:46 (1787851066) [ 4811.857085] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_hostid [ 4821.271743] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 4852.824582] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4852.851658] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4854.126128] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4854.158232] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4854.251785] Lustre: lustre-MDT0000: new disk, initializing [ 4854.342464] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4854.359332] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4858.673456] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 4868.710122] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4868.823659] Lustre: 166795:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4868.869528] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4868.873227] Lustre: Skipped 1 previous similar message [ 4868.956580] Lustre: lustre-MDT0001: new disk, initializing [ 4869.059153] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4869.108326] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4869.118827] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4873.498070] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 4881.888371] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4886.813492] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4891.876748] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4892.055732] Lustre: lustre-OST0000: new disk, initializing [ 4892.059945] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4892.068986] Lustre: 168767:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4892.128330] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4897.558633] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 4898.345634] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4898.359948] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4898.496914] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4905.144231] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4914.148942] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4914.152538] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4914.160095] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4914.166870] Lustre: Skipped 1 previous similar message [ 4914.191264] Lustre: Skipped 1 previous similar message [ 4919.240616] Lustre: server umount lustre-OST0000 complete [ 4924.392729] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4924.395421] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4924.403246] Lustre: Skipped 1 previous similar message [ 4928.523416] Lustre: server umount lustre-MDT0000 complete [ 4931.359275] LustreError: 168766:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787851201 with bad export cookie 3841326735948609192 [ 4931.361695] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4931.370633] LustreError: 168766:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4931.643928] Lustre: server umount lustre-MDT0001 complete [ 4941.190768] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [ 4943.597430] Key type lgssc unregistered [ 4943.864946] LNet: 170675:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4943.873773] LNetError: 170675:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4943.887796] LNet: Removed LNI 192.168.206.150@tcp [ 4944.777641] Key type .llcrypt unregistered [ 4944.781191] Key type ._llcrypt unregistered [ 4966.368370] Key type ._llcrypt registered [ 4966.379346] Key type .llcrypt registered [ 4966.558669] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 4967.610797] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4967.759134] alg: No test for adler32 (adler32-zlib) [ 4968.777365] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [ 4968.954184] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [ 4970.599188] Key type lgssc registered [ 4971.957298] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4981.940563] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4981.973992] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4983.511529] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4988.276572] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 4994.018573] LustreError: 172089:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4994.042627] LustreError: 172089:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 2 previous similar messages [ 4998.342503] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4999.012843] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5004.438716] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 5012.987643] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5019.104484] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5027.042644] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5027.395544] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5028.443227] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5034.956186] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 5042.800301] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5049.223852] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5049.248574] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5049.286247] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5049.291080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.301248] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5050.341873] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5050.360419] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5050.381249] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5050.384919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.418972] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5051.486466] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5051.505224] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5051.517724] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5051.523693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.536256] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5052.383908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.379959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.183739] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5054.188159] Lustre: Skipped 2 previous similar messages [ 5054.198976] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5054.206516] Lustre: Skipped 2 previous similar messages [ 5054.209585] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5054.213359] Lustre: Skipped 2 previous similar messages [ 5054.216746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.227180] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5054.232155] Lustre: Skipped 2 previous similar messages [ 5055.102822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.072664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.016196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.674096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.681956] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5058.688717] Lustre: Skipped 4 previous similar messages [ 5058.709969] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5058.721367] Lustre: Skipped 4 previous similar messages [ 5058.725568] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5058.732444] Lustre: Skipped 4 previous similar messages [ 5058.735914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.746115] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5058.752413] Lustre: Skipped 4 previous similar messages [ 5059.729792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.846965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.927755] LustreError: 172109:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5061.939850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.941850] LustreError: 172109:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5062.964415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.845286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.809439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.909116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.885471] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5066.893804] Lustre: Skipped 7 previous similar messages [ 5066.913693] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5066.937369] Lustre: Skipped 7 previous similar messages [ 5066.954956] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5066.964929] Lustre: Skipped 7 previous similar messages [ 5066.969732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.988434] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5067.002911] Lustre: Skipped 7 previous similar messages [ 5067.750959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.671171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.539965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.455385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.444208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.251586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.261500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.100836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.069821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.707641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.666323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.569809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.439357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.304463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.052172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.848114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.735529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.530498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.495797] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5083.499331] Lustre: Skipped 18 previous similar messages [ 5083.512484] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5083.524684] Lustre: Skipped 18 previous similar messages [ 5083.529026] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5083.534301] Lustre: Skipped 18 previous similar messages [ 5083.537113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.543066] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5083.548906] Lustre: Skipped 18 previous similar messages [ 5084.467809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.349262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.219093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.125711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.807234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.796728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.667852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.567683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.424858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.421326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.412985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.369238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.161193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.073469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.877089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.625651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.335842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.147193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.959132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.954311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.885566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.725531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.626723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.527945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.456881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.517708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.261766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.268731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.038961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.851524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.685675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.466963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.175726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.892341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.633742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.485968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.369595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.174741] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5116.179920] Lustre: Skipped 37 previous similar messages [ 5116.198903] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5116.205200] Lustre: Skipped 37 previous similar messages [ 5116.210254] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5116.213538] Lustre: Skipped 37 previous similar messages [ 5116.217619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.229820] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5116.239628] Lustre: Skipped 37 previous similar messages [ 5117.007985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.919645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.809820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.665860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.579838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.422596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.261756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.088599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.975160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.907574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.674417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.414253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.307902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.242370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.173522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.976644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.798341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.803439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.627535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.485783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.433296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.473770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.633489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.574881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.626898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.511202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.382888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.239066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.045463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.843126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.690515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.419755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.270129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.974805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.862367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.808498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.801191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.679287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.542152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.305256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.182694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.046465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.909898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.753709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.662555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.514062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.254301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.996625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.825671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.615541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.427640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.173204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.001451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.827395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.710190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.439860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.413023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.354702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.263671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.125875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.942057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.765562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.621963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.496847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.495240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.207406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.112546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.076581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.060015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.793279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.786089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.794993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.684440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.607418] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5180.611741] Lustre: Skipped 73 previous similar messages [ 5180.629514] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5180.638815] Lustre: Skipped 73 previous similar messages [ 5180.647284] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5180.652384] Lustre: Skipped 73 previous similar messages [ 5180.670287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.681922] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5180.689684] Lustre: Skipped 73 previous similar messages [ 5181.844478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.798326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.663464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.433254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.379755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.235508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.397973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.334797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.439405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.420170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.518261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.389505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.394448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.242515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.059984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.876747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.645570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.579164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.408989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.431691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.214883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.026820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.820493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.814295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.612460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.462654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.404323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.819531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.976502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.929874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.083326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.021396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.954772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.003684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.087810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.275712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.288083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.837732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.166576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.185817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.041644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.027270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.119247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.057413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.810572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.662955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.573518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.387781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.408830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.272106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.143390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.009927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.109787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.034932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.783225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.655883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.537058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.357326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.662161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.578591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.317680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.190617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.460543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.292730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.199546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.349776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.581601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.914818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.830955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.176083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.232374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.219702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.278650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.299822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.275707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.255524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.396446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.417279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.539174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.379932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.430546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.465307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.442863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.423204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.224598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.324860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.547562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.471457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.373499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.389603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.557887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.692390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.917140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.825794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.023199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.401789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.399135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.596883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.928968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.058556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.181664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.036572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.922625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.622268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.910826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.255663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.185296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.309610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.277135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.052115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.883136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.803153] LustreError: 172109:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5292.815685] LustreError: 172109:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5292.816180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.761290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.737960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.577606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.357455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.407707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.263931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.167511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.014782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.061609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.821432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.772815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.631614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.653498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.737600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.612811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.758219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.692552] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5308.696973] Lustre: Skipped 128 previous similar messages [ 5308.710953] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5308.734479] Lustre: Skipped 128 previous similar messages [ 5308.738414] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5308.742421] Lustre: Skipped 128 previous similar messages [ 5308.746121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.758486] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5308.768430] Lustre: Skipped 128 previous similar messages [ 5309.664741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.630805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.608996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.513686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.352046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.143508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.004973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.772748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.659638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.643534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.761483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.868133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.769461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.585111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.431679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.791507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.918224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.579797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.405820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.243199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.303381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.451757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.687964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.091671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.233897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.307862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.436069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.617460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.747616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.654218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.804361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.877507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.902327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.976698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.751619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.776704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.699589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.674891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.536220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.563682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.489848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.295849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.219584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.912549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.749113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.777387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.601966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.395683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.247065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.247602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.197925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.204918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.176332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.964231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.002194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.890825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.962735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.992709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.106263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.421354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.570273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.739538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.582837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.700828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.886816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.884248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.991990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.859521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.842945] LustreError: 172109:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5376.851674] LustreError: 172109:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5376.878838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.834597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.890758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.871832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.859255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.758689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.614233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.437537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.270821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.078852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.962869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.160975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.271562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.477948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.811388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.718068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.734506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.607845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.592892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.601745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.311153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.340030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.436066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.530380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.399584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.743456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.246939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.004715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.424486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.770733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.492932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.913843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.439222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.927863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.216353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.653566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.638371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.753620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.004988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.747670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.048530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.353766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.646506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.748377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.134437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.356325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.266717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.389525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.361047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.648706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.552373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.332604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.148549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.101719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.109691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.004252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.879615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.020939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.793741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.945401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.170395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.199600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.520620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.475866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.621264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.873912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.094482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.426250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.768328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.250166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.735813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.197281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.891993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.615273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.772824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.180784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.586472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.128654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.093863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.714675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.265651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.513224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.688309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.587199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.860128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.392823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.646753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.951232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.166793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.354409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.390464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.422254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.442974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.376260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.275943] LustreError: 172109:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5487.294908] LustreError: 172109:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5487.311348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.185217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.142219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.135720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.103738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.227633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.387875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.367385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.342407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.314717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.287725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.362445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.294514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.127247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.004485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.914837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.755418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.801444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.875942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.937659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.875643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.011907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.025375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.240455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.252502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.166294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.034473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.191241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.457584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.964913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.346949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.495672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.587208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.866391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.087961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.587553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.622085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.882591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.199398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.269752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.256573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.091602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.118872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.941768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.704848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.552053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.476453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.379839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.243066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.270930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.323218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.206553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.426507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.291776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.092675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.818119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.811895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.691817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.648842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.477283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.412188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.315651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.191631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.075426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.934819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.093800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.983179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.944507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.812287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.816216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.653570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.553398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.451531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.329360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.190641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.089862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.868344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.674667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.505840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.364859] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5565.379098] Lustre: Skipped 241 previous similar messages [ 5565.399516] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5565.410595] Lustre: Skipped 241 previous similar messages [ 5565.414524] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5565.424555] Lustre: Skipped 241 previous similar messages [ 5565.429812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.438455] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5565.444177] Lustre: Skipped 241 previous similar messages [ 5566.195050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.043743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.963883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.920284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.963058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.893302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.712728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.661125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.546243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.614689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.500869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.275672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.115659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.888086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.729475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.814555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.733463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.970342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.059688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.135269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.436667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.734634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.027152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.165947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.058504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.230041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.470572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.552432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.527373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.555730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.957672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.226639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.139740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.329691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.227313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.269905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.157971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.166900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.124068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.967705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.377178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.121973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.961334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.190488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.361176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.234598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.291934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.568580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.580457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.551100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.512938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.482831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.441371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.344334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.321604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.230655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.151529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.142985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.942838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.954317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.945523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.999846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.883679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.949106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.861632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.885447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.719952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.348776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.172629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.038319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.998500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.823181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.743963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.633017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.366212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.276363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.001229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.884368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.080728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.999395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.844536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.873210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.646412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.543036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.448822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.519641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.386064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.313602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.318244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.365035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.427223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.449372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.427931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.460229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.557195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.444734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.535480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.494590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.497630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.327843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.251629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.016772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.901251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.790724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.891146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.029867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.110088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.846696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.868246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.997670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.059258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.022073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.040501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.034914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.968857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.800737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.706850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.724355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.915671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.037829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.018260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.134542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.090472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.896699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.711943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.731963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.532700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.969842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.014179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.195893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.183072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.201120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.284463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.364908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.753608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.993114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.241111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.375475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.414470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.536969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.561065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.026301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.220630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.274229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.422664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.665331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.750573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.902038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.899189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.780727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.569049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.468603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.426266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.634169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.497983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.410172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.558318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.507446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.621479] LustreError: 172109:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5724.646094] LustreError: 172109:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5724.662054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.613499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.528134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.322719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.264134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.196085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.302207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.384961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.749505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.220196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.384717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.302131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.229427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.219648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.373095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.362830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.346885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.294197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.183507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.163678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.038670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.969576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.004414] LustreError: 172109:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5747.012741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.019762] LustreError: 172109:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5748.061945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.905883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.733688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.777237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.586866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.739228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.863235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.789723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.654818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.962354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.808945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.153175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.241738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.375714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.289029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.450383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.664918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.731209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.742036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.676377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.821533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.488277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.279123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.234101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.411785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.313832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.342935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.211326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.019359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.807678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.866347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.940034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.749132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.776974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.630620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.466127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.405739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.579648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.483864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.488033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.659281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.481316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.396055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.335174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.370293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.320633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.323049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.451867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.331583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.257541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.163387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.051264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.772953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.865380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.691519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.564566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.260305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.086091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.055495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.038181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.756478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.484936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.168140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.993060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.775882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.578239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.300515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.175961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.994055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.801053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.524812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.390860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.139975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.076075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.061635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.936162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.700739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.535531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.612444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.344458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.394168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.284396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.107338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.120618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.046960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.956679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.731208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.666495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.130989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.106627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.036351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.978878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.193231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.485712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.640275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.876617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.713099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.573466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.798499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.066373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.039273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.320621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.601641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.528389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.292046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.333218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.208929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.042287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.753056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.838504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.051179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.174401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.157069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.181173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.049037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.156534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.071673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.217597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.202123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.297923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.267906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.238901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.527381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.917255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.900230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.280127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.456036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.675331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.735659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.083872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.504541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.449376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.593462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.559441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.958227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.985262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.391962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.290836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.135774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.901807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.331750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.146472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.687917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.575942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.910950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.201555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.498499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.742552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.304163] LustreError: 172109:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5902.326809] LustreError: 172109:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5902.354736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.134807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.365784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.978812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.238861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.502335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.534493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.579734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.667662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.898993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.140602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.283410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.259699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.235175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.130172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.171245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.074729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.963134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.832597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.929676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.107777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.174732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.315060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.514268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.536176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.592128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.601389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.497855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.445061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.392536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.267355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.153528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.066581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.028481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.821867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.590426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.377816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.288709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.075213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.916259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.705256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.555970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.317793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.089817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.958734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.892639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.826383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.779741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.645387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.761142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.700220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.625618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.519800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.394569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.327949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.185617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.202867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.100122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.122697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.306080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.412327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.527757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.262131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.168207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.230428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.249151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.230954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.297883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.461688] LustreError: 172109:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5970.487095] LustreError: 172109:0:(osp_precreate.c:655:osp_precreate_send()) Skipped 1 previous similar message [ 5970.496687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.501762] LustreError: 172109:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5970.501772] LustreError: 172109:0:(osp_precreate.c:1359:osp_precreate_thread()) Skipped 1 previous similar message [ 5971.898216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.111670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.265988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.621752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.954837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.130302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.391500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.417034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.456877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.417915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.241657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.126457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.975048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.831824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.737410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.693809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.612162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.663212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.612823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.539693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.453892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.479445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.399888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.374797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.179457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.119633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.028122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.900522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.840057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.802954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.796229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.960290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.939918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.143684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.133945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.314798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.305881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.367874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.374232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.296714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.252508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.338333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.156913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.028425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.893523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.835116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.650957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.597907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.425796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.522355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.693178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.763696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.642591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.560592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.601222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.437423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.394591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.252719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.148179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.039633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.865711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.855289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.862557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.980778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.791867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.801792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.622334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.589292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.621342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.565108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.329041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.282220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.141337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.995731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.975785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.143398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.259603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.341508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.374863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.417686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.466457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.881672] LustreError: 173910:0:(ofd_dev.c:1783:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6505.897655] LustreError: 172109:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7168.490617] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7168.498874] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7168.520415] Lustre: Skipped 479 previous similar messages [ 7168.533958] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7171.056649] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 7171.059927] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7171.070890] LustreError: Skipped 4 previous similar messages [ 7172.577923] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7172.586538] Lustre: Skipped 1 previous similar message [ 7174.614929] Lustre: server umount lustre-OST0000 complete [ 7180.262864] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7180.276440] LustreError: Skipped 3 previous similar messages [ 7180.283233] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7184.512367] Lustre: server umount lustre-MDT0000 complete [ 7187.939633] LustreError: 172085:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7187.957426] LustreError: 172085:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [ 7188.087853] LustreError: 172070:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787853458 with bad export cookie 6853021860423161928 [ 7188.096388] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7188.099277] LustreError: 172070:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7188.423140] Lustre: server umount lustre-MDT0001 complete [ 7198.946940] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [ 7201.813701] Key type lgssc unregistered [ 7202.204518] LNet: 227364:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7202.211743] LNetError: 227364:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7202.250650] LNet: Removed LNI 192.168.206.150@tcp [ 7203.157077] Key type .llcrypt unregistered [ 7203.162987] Key type ._llcrypt unregistered [ 7229.076147] Key type ._llcrypt registered [ 7229.081414] Key type .llcrypt registered [ 7229.193340] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 7230.559724] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7230.622203] alg: No test for adler32 (adler32-zlib) [ 7231.723233] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [ 7231.932368] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [ 7233.567156] Key type lgssc registered [ 7234.598446] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7242.095066] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7242.132770] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7243.641461] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7247.770271] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 7253.998434] LustreError: 228782:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7254.033922] LustreError: 228782:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 2 previous similar messages [ 7256.416259] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7256.717824] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7261.281687] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 7269.671827] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7275.596702] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7283.166738] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7283.564238] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7288.722945] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7289.596404] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 7296.758305] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7305.766813] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7307.749826] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7307.778108] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7307.786357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7307.813749] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7307.813777] LustreError: 228801:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7309.882864] Lustre: lustre-OST0000: Not available for connect from 192.168.206.50@tcp (stopping) [ 7312.353495] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7312.366156] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7312.378733] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7314.404133] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7314.407419] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7315.731639] Lustre: server umount lustre-OST0000 complete [ 7327.847479] LustreError: 228801:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7349.877394] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7350.180582] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7354.296519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7354.313819] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7355.781865] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 7362.658548] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7364.586323] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7364.611550] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7367.211343] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 3 sec [ 7372.795269] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8ee8c29ac800.ost_server_uuid 50 [ 7374.234027] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8ee8c29ac800.ost_server_uuid in FULL state after 0 sec [ 7379.433349] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7379.443691] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7379.454751] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7379.463716] Lustre: Skipped 2 previous similar messages [ 7380.966785] Lustre: server umount lustre-OST0000 complete [ 7385.061237] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7385.065285] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7385.082198] Lustre: Skipped 2 previous similar messages [ 7385.096343] Lustre: Skipped 2 previous similar messages [ 7395.296830] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7395.301735] Lustre: Skipped 3 previous similar messages [ 7399.394045] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7399.670236] Lustre: server umount lustre-MDT0000 complete [ 7403.209695] LustreError: 228761:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787853673 with bad export cookie 6214768593079293021 [ 7403.229227] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7403.547401] Lustre: server umount lustre-MDT0001 complete [ 7415.107687] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [ 7417.772269] Key type lgssc unregistered [ 7418.173911] LNet: 234006:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7418.199307] LNetError: 234006:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7419.238760] LNet: Removed LNI 192.168.206.150@tcp [ 7420.059751] Key type .llcrypt unregistered [ 7420.061907] Key type ._llcrypt unregistered [ 7450.046778] Key type ._llcrypt registered [ 7450.049204] Key type .llcrypt registered [ 7450.159404] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 7451.219763] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7451.227409] alg: No test for adler32 (adler32-zlib) [ 7452.214790] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [ 7452.530853] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [ 7454.199269] Key type lgssc registered [ 7455.353905] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7463.466962] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7463.485763] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7464.970481] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7469.235880] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 7474.163888] LustreError: 235733:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7474.179762] LustreError: 235733:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 2 previous similar messages [ 7476.299593] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7476.677617] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7480.977760] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 7483.782836] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7489.466136] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7489.762731] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7490.831503] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7495.320503] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 7503.480871] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7503.697663] Lustre: lustre-OST0001: new disk, initializing [ 7503.700702] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7503.711665] Lustre: 238039:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7503.821157] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7509.351937] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 7511.621875] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7511.631167] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7511.715094] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7519.055922] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7531.036215] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7543.268713] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7543.279924] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7543.290510] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7545.464351] Lustre: server umount lustre-MDT0000 complete [ 7547.880388] LustreError: 235733:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7547.884952] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7547.918968] Lustre: Skipped 2 previous similar messages [ 7548.679301] LustreError: 235715:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787853818 with bad export cookie 8503283277395978961 [ 7548.679782] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7548.684502] LustreError: 235715:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7548.916361] Lustre: server umount lustre-MDT0001 complete [ 7562.363535] Lustre: server umount lustre-OST0000 complete [ 7576.520771] Lustre: server umount lustre-OST0001 complete [ 7601.888781] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 7612.747132] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7613.149039] LustreError: 241151:0:(ldlm_lib.c:1200:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7613.167893] LustreError: 241151:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 3 previous similar messages [ 7613.260142] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7618.537167] LustreError: 241152:0:(ldlm_lib.c:1200:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7618.751506] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 7623.655178] LustreError: 241151:0:(ldlm_lib.c:1200:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7628.562970] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7629.151409] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7635.619992] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 7638.927130] Lustre: 242296:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7647.090274] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7647.575874] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7653.740145] LustreError: 242648:0:(ldlm_lib.c:1200:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7653.760895] LustreError: 242648:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [ 7654.672107] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 7658.899905] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 7664.674468] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7670.276025] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7671.974753] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 7679.667988] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7683.674225] Lustre: 244168:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7695.853536] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7695.872162] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7695.873825] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7695.877069] Lustre: Skipped 3 previous similar messages [ 7695.893688] Lustre: Skipped 3 previous similar messages [ 7699.657648] Lustre: server umount mylustre-MDT0000 complete [ 7700.973958] LustreError: 241147:0:(ldlm_lib.c:1200:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7701.005641] LustreError: 241147:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 7 previous similar messages [ 7703.099114] LustreError: 241127:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787853973 with bad export cookie 8503283277395981768 [ 7703.100530] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7703.129446] LustreError: 241127:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7703.683474] Lustre: server umount mylustre-MDT0001 complete [ 7718.262960] Lustre: server umount mylustre-OST0000 complete [ 7721.439235] Lustre: 234997:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787853975/real 1787853975] req@ffff9647aba3c780 x1874700504553472/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787853991 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7721.484181] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7722.774794] Lustre: server umount mylustre-OST0001 complete [ 7745.180518] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 7756.205213] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7756.631118] LustreError: 246475:0:(ldlm_lib.c:1200:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7756.720589] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7756.724494] Lustre: Skipped 1 previous similar message [ 7761.844215] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 7771.088700] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7776.661429] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 7779.712162] Lustre: 247615:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7787.411261] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7795.864498] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 7801.336873] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 7806.425952] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7812.115794] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 7813.619651] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 7822.831976] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7827.834145] Lustre: 249491:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7837.669446] Lustre: tfs-MDT0000-lwp-OST0000: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7837.684396] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7837.687103] Lustre: Skipped 1 previous similar message [ 7837.727658] Lustre: Skipped 1 previous similar message [ 7839.513880] Lustre: server umount tfs-MDT0000 complete [ 7842.785204] LustreError: 248582:0:(ldlm_lib.c:1200:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7842.831599] LustreError: 248582:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 10 previous similar messages [ 7843.792882] LustreError: 246450:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787854113 with bad export cookie 8503283277395983658 [ 7843.797922] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7843.802493] LustreError: 246450:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7844.367391] Lustre: server umount tfs-MDT0001 complete [ 7858.464841] Lustre: server umount tfs-OST0000 complete [ 7873.233781] Lustre: server umount tfs-OST0001 complete [ 7895.931906] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 7907.560643] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7908.177494] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7908.186220] Lustre: Skipped 3 previous similar messages [ 7913.120582] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 7922.447163] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7928.246212] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 7931.100823] Lustre: 252886:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7939.374067] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7946.266168] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 7953.411579] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 7955.450328] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7961.094611] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 7962.227766] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 7969.644554] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7973.432172] Lustre: 254754:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7980.794454] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 14:10:48 (1787854248) [ 7996.899690] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7996.914892] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7996.922493] Lustre: Skipped 4 previous similar messages [ 7996.944704] Lustre: Skipped 5 previous similar messages [ 8002.016934] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8002.043466] Lustre: Skipped 3 previous similar messages [ 8002.944509] Lustre: server umount lustre-MDT0000 complete [ 8007.154222] LustreError: 251743:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8007.190676] LustreError: 251743:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 10 previous similar messages [ 8007.355193] LustreError: 251719:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787854277 with bad export cookie 8503283277395985324 [ 8007.369737] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8007.382364] LustreError: 251719:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8012.289039] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8012.304145] Lustre: Skipped 1 previous similar message [ 8014.013917] Lustre: server umount lustre-MDT0001 complete [ 8024.376721] Lustre: server umount lustre-OST0000 complete [ 8034.473785] Lustre: server umount lustre-OST0001 complete [ 8041.930114] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_hostid [ 8051.558966] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 8092.758659] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8093.103279] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8093.137628] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8093.206959] Lustre: lustre-MDT0000: new disk, initializing [ 8093.312860] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8093.319497] Lustre: Skipped 3 previous similar messages [ 8093.334378] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8098.241616] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 8107.624264] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8107.713919] Lustre: 257941:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8107.762988] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8107.766081] Lustre: Skipped 1 previous similar message [ 8107.827120] Lustre: lustre-MDT0001: new disk, initializing [ 8107.997738] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8108.008781] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8113.140918] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 8124.746362] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8125.156729] Lustre: lustre-OST0000: new disk, initializing [ 8125.159948] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8125.168740] Lustre: 259523:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8126.718483] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8126.735911] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8126.870877] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8132.795382] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 8144.630190] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8144.992610] Lustre: lustre-OST0001: new disk, initializing [ 8145.007266] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8145.026846] Lustre: 260393:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8147.005438] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8147.025551] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8147.127726] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8152.307866] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 8164.088225] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 14:13:51 (1787854431) [ 8180.202604] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 14:14:07 (1787854447) [ 8198.638151] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8198.642300] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8198.649199] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8198.663552] Lustre: Skipped 4 previous similar messages [ 8198.691289] LustreError: Skipped 1 previous similar message [ 8201.564570] Lustre: server umount lustre-OST0000 complete [ 8212.649640] Lustre: server umount lustre-MDT0000 complete [ 8216.031404] LustreError: 257933:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787854485 with bad export cookie 8503283277395986878 [ 8216.033076] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8216.042156] LustreError: 257933:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8230.367122] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8230.607413] Lustre: server umount lustre-MDT0001 complete [ 8243.692835] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [ 8313.819206] Lustre: server umount lustre-OST0001 complete [ 8321.097677] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_hostid [ 8339.513724] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 8374.521897] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8374.760936] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8374.786565] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8374.864358] Lustre: lustre-MDT0000: new disk, initializing [ 8374.936662] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8374.945056] Lustre: Skipped 3 previous similar messages [ 8374.958401] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8379.486368] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 8389.816346] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8389.912516] Lustre: 265242:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8390.215818] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8395.222447] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 8405.073972] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8410.869657] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8418.884807] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8419.441086] Lustre: lustre-OST0000: new disk, initializing [ 8419.445802] Lustre: Skipped 1 previous similar message [ 8419.467070] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8419.486023] Lustre: Skipped 2 previous similar messages [ 8419.493740] Lustre: 267211:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8421.425164] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8421.429837] Lustre: Skipped 1 previous similar message [ 8421.434540] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8421.571586] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8425.809549] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 8435.615062] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8447.456611] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8447.460304] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8447.468195] LustreError: Skipped 1 previous similar message [ 8447.479249] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8447.504280] Lustre: Skipped 5 previous similar messages [ 8447.506435] Lustre: Skipped 8 previous similar messages [ 8449.922687] Lustre: server umount lustre-OST0000 complete [ 8456.687894] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8460.406319] Lustre: server umount lustre-MDT0000 complete [ 8462.817128] LustreError: 265252:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8462.840289] LustreError: 265252:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 14 previous similar messages [ 8464.094083] LustreError: 266037:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787854734 with bad export cookie 8503283277395988341 [ 8464.095611] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8464.108134] LustreError: 266037:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8464.480267] Lustre: server umount lustre-MDT0001 complete [ 8474.456630] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [ 8477.271215] Key type lgssc unregistered [ 8477.586613] LNet: 268973:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8477.597813] LNetError: 268973:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8477.617565] LNet: Removed LNI 192.168.206.150@tcp [ 8478.437158] Key type .llcrypt unregistered [ 8478.444455] Key type ._llcrypt unregistered [ 8506.666328] Key type ._llcrypt registered [ 8506.673672] Key type .llcrypt registered [ 8506.773381] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 8507.656250] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8507.666640] alg: No test for adler32 (adler32-zlib) [ 8508.691600] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [ 8508.888025] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [ 8510.561975] Key type lgssc registered [ 8511.620729] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8518.946810] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8518.961765] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8520.465514] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8524.866276] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 8530.915741] LustreError: 270436:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8530.939736] LustreError: 270436:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 2 previous similar messages [ 8533.034730] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8533.531228] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8538.280273] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 8545.874239] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8551.647937] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8558.778432] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8559.090063] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8565.094545] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 8565.242819] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8572.407386] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8573.513273] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8579.558763] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8579.572176] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8579.602416] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8580.581531] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8580.583091] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8580.593476] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8580.621013] Lustre: Skipped 1 previous similar message [ 8581.885493] Lustre: server umount lustre-OST0000 complete [ 8585.699546] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8585.716234] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8585.721726] Lustre: Skipped 1 previous similar message [ 8585.736800] Lustre: Skipped 1 previous similar message [ 8590.824699] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8590.839945] Lustre: Skipped 1 previous similar message [ 8593.546600] Lustre: server umount lustre-MDT0000 complete [ 8595.954825] LustreError: 270436:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8595.975197] LustreError: 270436:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [ 8596.587677] LustreError: 270417:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787854866 with bad export cookie 3663640143656020240 [ 8596.591158] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8596.604100] LustreError: 270417:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8596.952807] Lustre: server umount lustre-MDT0001 complete [ 8607.324290] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [ 8609.782061] Key type lgssc unregistered [ 8610.095277] LNet: 274025:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8610.106533] LNetError: 274025:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8610.118883] LNet: Removed LNI 192.168.206.150@tcp [ 8610.976974] Key type .llcrypt unregistered [ 8610.979461] Key type ._llcrypt unregistered [ 8627.376727] Key type ._llcrypt registered [ 8627.379836] Key type .llcrypt registered [ 8627.470663] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 8628.373142] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8628.395046] alg: No test for adler32 (adler32-zlib) [ 8629.468398] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [ 8629.686448] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [ 8631.327356] Key type lgssc registered [ 8632.262492] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8640.331835] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8640.356018] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8655.903430] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8661.023345] LustreError: 275219:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.150@tcp: failed processing log, type 4: rc = -110 [ 8687.777552] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8693.074095] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 8700.607117] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8702.613790] Lustre: server umount lustre-OST0000 complete [ 8717.243627] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [ 8719.920561] Key type lgssc unregistered [ 8720.200914] LNet: 276854:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8720.218206] LNetError: 276854:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8720.237955] LNet: Removed LNI 192.168.206.150@tcp [ 8721.004518] Key type .llcrypt unregistered [ 8721.008970] Key type ._llcrypt unregistered [ 8754.631987] Key type ._llcrypt registered [ 8754.633604] Key type .llcrypt registered [ 8754.744278] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 8755.764058] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8755.782919] alg: No test for adler32 (adler32-zlib) [ 8756.749765] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [ 8756.919842] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [ 8758.575156] Key type lgssc registered [ 8759.796745] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8770.952994] loop0: detected capacity change from 0 to 204800000 [ 8771.074609] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8771.085358] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8771.350804] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8772.227989] loop0: detected capacity change from 0 to 204800000 [ 8772.371131] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8772.382161] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8772.681569] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8773.483911] loop0: detected capacity change from 0 to 204800000 [ 8773.676342] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8773.687359] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8773.778975] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8774.816211] loop0: detected capacity change from 0 to 204800000 [ 8775.029247] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8776.021921] loop0: detected capacity change from 0 to 204800000 [ 8776.044766] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8779.943125] loop0: detected capacity change from 0 to 204800000 [ 8779.984295] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8784.362863] loop0: detected capacity change from 0 to 204800000 [ 8784.400491] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8788.769330] loop0: detected capacity change from 0 to 204800000 [ 8788.890736] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8793.144970] loop0: detected capacity change from 0 to 204800000 [ 8793.232852] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8793.302608] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8793.414919] Lustre: 280251:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8793.430370] Lustre: 280251:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8793.449266] Lustre: 280251:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8793.477601] Lustre: 280251:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8793.489953] Lustre: 280251:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8793.502275] Lustre: 280251:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8793.520902] Lustre: 280251:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8793.542818] Lustre: 280251:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8793.564925] Lustre: 280251:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8793.583395] Lustre: 280251:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8793.601842] Lustre: 280251:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8793.613172] Lustre: 280251:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8793.626308] Lustre: 280251:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8793.645843] Lustre: 280251:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8800.522826] Lustre: server umount MGS complete [ 8801.429609] loop0: detected capacity change from 0 to 204800000 [ 8801.497190] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8801.921477] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8801.966249] Lustre: lustre-MDT0000: Aborting client recovery [ 8803.051472] loop1: detected capacity change from 0 to 204800000 [ 8803.225868] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8803.373248] Lustre: 280683:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8803.401211] Lustre: 280683:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8803.426991] Lustre: 280683:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8803.450802] Lustre: 280683:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8803.477690] Lustre: 280683:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8803.508849] Lustre: 280683:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8803.525638] Lustre: 280683:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8803.550072] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8803.854616] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8803.926283] Lustre: lustre-MDT0001: Aborting client recovery [ 8804.925049] loop2: detected capacity change from 0 to 204800000 [ 8805.139361] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8805.217940] Lustre: 280762:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8805.235414] Lustre: 280762:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8805.244959] Lustre: 280762:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8805.255229] Lustre: 280762:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8805.271767] Lustre: 280762:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8805.282709] Lustre: 280762:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8805.301770] Lustre: 280762:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8805.466326] Lustre: Found index 0 for lustre-OST0000, updating log [ 8805.535513] Lustre: 280782:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8805.625429] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8805.672573] Lustre: lustre-OST0000: Aborting recovery [ 8806.816186] loop3: detected capacity change from 0 to 204800000 [ 8807.021224] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8807.116243] Lustre: 280847:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8807.134339] Lustre: 280847:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8807.158531] Lustre: 280847:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8807.170750] Lustre: 280847:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8807.190384] Lustre: 280847:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8807.215781] Lustre: 280847:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8807.236929] Lustre: 280847:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8807.264404] Lustre: Found index 1 for lustre-OST0001, updating log [ 8807.287248] Lustre: 280854:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8807.468261] Lustre: lustre-OST0001: Aborting recovery [ 8811.514294] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8811.556983] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8811.608638] LustreError: 280627:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8811.652463] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8811.663394] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8813.603488] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8813.611263] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8815.749207] LustreError: 280594:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 13211us: rc = -2 [ 8816.303094] LustreError: 280595:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 394us: rc = -2 [ 8816.317524] LustreError: 280595:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 8829.415991] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8829.441120] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8829.465814] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8834.073231] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8834.073473] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8834.085377] Lustre: Skipped 2 previous similar messages [ 8834.108204] Lustre: Skipped 2 previous similar messages [ 8835.373295] Lustre: server umount lustre-MDT0000 complete [ 8836.480317] LustreError: 280577:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787855106 with bad export cookie 12317943362807854786 [ 8836.499791] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8839.137582] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8839.140248] LustreError: 280591:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8839.140405] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8839.140408] Lustre: Skipped 1 previous similar message [ 8839.162964] Lustre: Skipped 1 previous similar message [ 8839.215018] LustreError: 280591:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 3 previous similar messages [ 8841.191119] LustreError: 281113:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8841.191641] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8841.202370] LustreError: 281113:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [ 8841.213630] Lustre: Skipped 2 previous similar messages [ 8842.851523] Lustre: server umount lustre-MDT0001 complete [ 8858.079268] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8858.255599] Lustre: server umount lustre-OST0000 complete [ 8865.233449] Lustre: server umount lustre-OST0001 complete [ 8873.604977] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 14:25:41 (1787855141) [ 8899.860921] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_hostid [ 8910.404392] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 8946.858396] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8947.067199] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8947.089330] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8947.142464] Lustre: lustre-MDT0000: new disk, initializing [ 8947.195225] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8947.202050] Lustre: Skipped 1 previous similar message [ 8947.220244] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8951.585714] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 8961.467705] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8961.593036] Lustre: 284922:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8961.633128] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8961.636207] Lustre: Skipped 1 previous similar message [ 8961.708329] Lustre: lustre-MDT0001: new disk, initializing [ 8961.800233] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8961.845661] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8961.858320] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8965.544389] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 8974.821828] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8980.475773] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8988.093263] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8988.397751] Lustre: lustre-OST0000: new disk, initializing [ 8988.405502] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8988.418641] Lustre: 286891:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8988.551842] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8994.416920] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8994.443098] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8994.679898] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8995.582193] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 9004.931318] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9009.784714] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9018.869325] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9027.872283] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9076.705609] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9076.711816] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9076.719032] LustreError: Skipped 1 previous similar message [ 9076.722344] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9076.730445] Lustre: Skipped 1 previous similar message [ 9076.751671] Lustre: Skipped 1 previous similar message [ 9079.943771] Lustre: server umount lustre-OST0000 complete [ 9084.899231] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9084.911581] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9084.934057] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9090.890716] Lustre: server umount lustre-MDT0000 complete [ 9092.067545] LustreError: 285765:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9092.116557] LustreError: 285765:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [ 9096.120427] LustreError: 284913:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787855366 with bad export cookie 12317943362807857488 [ 9096.122155] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9096.133725] LustreError: 284913:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 9096.699691] Lustre: server umount lustre-MDT0001 complete [ 9105.107332] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9110.099177] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 9114.837324] Lustre: server umount MGS complete [ 9126.750751] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9127.227363] LustreError: 290813:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9127.243410] LustreError: 290826:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9127.298499] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9131.410902] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 9132.518160] LustreError: 290814:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9138.233958] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9142.754822] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 9149.438666] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9156.493628] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9164.730362] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9165.273930] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9165.280371] Lustre: Skipped 1 previous similar message [ 9172.912719] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 9181.366382] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9212.712019] Lustre: server umount lustre-OST0000 complete [ 9219.555579] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9219.558760] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9219.572112] Lustre: Skipped 2 previous similar messages [ 9219.580635] Lustre: Skipped 3 previous similar messages [ 9226.864825] Lustre: server umount lustre-MDT0000 complete [ 9229.802063] LustreError: 292656:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9229.835436] LustreError: 292656:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 2 previous similar messages [ 9230.815567] Lustre: 278212:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787855484/real 1787855484] req@00000000091c7f13 x1874701871927936/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1787855500 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'osp-pre-0-1.0' uid:0 gid:0 projid:4294967295 [ 9230.839790] Lustre: 278212:0:(client.c:2503:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 9231.531544] LustreError: 290793:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787855501 with bad export cookie 12317943362807859329 [ 9231.553887] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9231.556792] LustreError: 290793:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9232.017285] Lustre: server umount lustre-MDT0001 complete [ 9242.439543] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [ 9245.016413] Key type lgssc unregistered [ 9245.312511] LNet: 294763:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9245.317833] LNetError: 294763:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9246.373373] LNet: Removed LNI 192.168.206.150@tcp [ 9247.303977] Key type .llcrypt unregistered [ 9247.307460] Key type ._llcrypt unregistered [ 9286.271678] Key type ._llcrypt registered [ 9286.273354] Key type .llcrypt registered [ 9286.389736] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_hostid [ 9303.272940] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 9304.668547] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9304.706752] alg: No test for adler32 (adler32-zlib) [ 9305.829923] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [ 9306.132256] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [ 9307.791176] Key type lgssc registered [ 9309.100452] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9345.663671] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9345.698489] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9347.021639] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9347.059126] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9347.162681] Lustre: lustre-MDT0000: new disk, initializing [ 9347.250309] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9347.272250] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9352.378063] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 9363.906617] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9364.044148] Lustre: 298723:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9364.128221] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9364.133144] Lustre: Skipped 1 previous similar message [ 9364.220197] Lustre: lustre-MDT0001: new disk, initializing [ 9364.358157] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9364.426304] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9364.438656] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9369.484783] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 9381.182362] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9387.287451] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9394.416331] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9394.744517] Lustre: lustre-OST0000: new disk, initializing [ 9394.747850] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9394.753616] Lustre: 300695:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9394.855458] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9401.223812] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 9403.478681] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9403.485932] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9403.581493] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9410.881878] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9417.415799] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9428.669432] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9432.805711] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9437.624852] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9485.792849] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9485.795756] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9485.797749] LustreError: Skipped 1 previous similar message [ 9485.799355] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9485.813180] Lustre: Skipped 1 previous similar message [ 9485.830097] Lustre: Skipped 1 previous similar message [ 9488.869710] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9488.878720] Lustre: Skipped 1 previous similar message [ 9490.942151] Lustre: server umount lustre-OST0000 complete [ 9497.568796] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9497.581658] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9497.614093] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9499.104467] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9501.739977] Lustre: server umount lustre-MDT0000 complete [ 9506.251746] LustreError: 298715:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787855776 with bad export cookie 185231916456691070 [ 9506.253990] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9506.271554] LustreError: 298715:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9506.798137] Lustre: server umount lustre-MDT0001 complete [ 9515.914178] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9520.187614] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 9524.817103] Lustre: server umount MGS complete [ 9537.040869] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9537.533430] LustreError: 304512:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9537.557031] LustreError: 304512:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 3 previous similar messages [ 9537.569300] LustreError: 304525:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9537.666090] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9542.647228] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 9543.136413] LustreError: 304513:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9547.236259] LustreError: 304512:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9552.104071] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9552.356706] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 9552.362855] Lustre: Skipped 2 previous similar messages [ 9552.573970] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9557.272859] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 9567.203939] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9575.451566] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9583.108952] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9583.599588] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9591.314380] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 9598.781596] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9629.669667] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9629.679333] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9629.679987] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9629.691924] Lustre: Skipped 1 previous similar message [ 9639.906529] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9639.919214] Lustre: Skipped 4 previous similar messages [ 9642.975140] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9643.115689] Lustre: server umount lustre-OST0000 complete [ 9649.643475] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9649.653177] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9655.729641] Lustre: server umount lustre-MDT0000 complete [ 9659.340869] LustreError: 307350:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787855929 with bad export cookie 185231916456692911 [ 9659.351039] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9659.351404] LustreError: 307350:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9659.632222] Lustre: server umount lustre-MDT0001 complete [ 9669.824901] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [ 9672.326660] Key type lgssc unregistered [ 9672.664883] LNet: 308464:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9672.671166] LNetError: 308464:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9673.703042] LNet: Removed LNI 192.168.206.150@tcp [ 9674.599963] Key type .llcrypt unregistered [ 9674.602506] Key type ._llcrypt unregistered [ 9693.906391] Key type ._llcrypt registered [ 9693.908702] Key type .llcrypt registered [ 9693.991601] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_hostid [ 9708.760401] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 9710.264929] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9710.622442] alg: No test for adler32 (adler32-zlib) [ 9711.571399] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [ 9711.756585] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [ 9713.424140] Key type lgssc registered [ 9714.799747] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9739.734476] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [ 9755.438654] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9755.486128] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9756.858473] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9756.918356] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9757.014143] Lustre: lustre-MDT0000: new disk, initializing [ 9757.107993] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9757.142663] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9761.728346] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 9773.095073] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9773.447168] Lustre: lustre-OST0000: new disk, initializing [ 9773.454153] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9773.456984] Lustre: Skipped 1 previous similar message [ 9773.467291] Lustre: 312516:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9773.547028] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9780.839886] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [ 9781.856769] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9781.874431] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9781.901473] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10667.488519] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10667.501842] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10672.336604] Lustre: server umount lustre-MDT0000 complete [10676.112716] LustreError: 311701:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787856946 with bad export cookie 10875767730073989331 [10676.125659] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10676.133219] LustreError: 311701:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10676.318982] Lustre: server umount lustre-OST0000 complete [10700.496791] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_hostid [10709.454332] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [10743.831991] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10744.029935] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10744.065538] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10744.134704] Lustre: lustre-MDT0000: new disk, initializing [10744.194340] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10744.225358] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10748.579381] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [10759.863217] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10759.986916] Lustre: 316862:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10760.089276] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10760.097358] Lustre: Skipped 1 previous similar message [10760.165563] Lustre: lustre-MDT0001: new disk, initializing [10760.300721] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10760.346501] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10760.362548] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10764.892515] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [10774.317427] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10780.542650] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10788.225357] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10788.412830] Lustre: lustre-OST0000: new disk, initializing [10788.416718] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10788.436502] Lustre: 318831:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10788.581046] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10794.820467] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [10795.093583] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10795.105295] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10795.261419] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10805.307888] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10812.229030] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10812.417812] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10817.741071] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10818.054410] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10821.093638] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10821.095589] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10821.116521] LustreError: Skipped 1 previous similar message [10821.117607] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10821.140261] Lustre: Skipped 1 previous similar message [10821.172464] Lustre: Skipped 1 previous similar message [10826.210594] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10831.338497] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10831.346969] Lustre: Skipped 2 previous similar messages [10834.399269] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10834.667495] Lustre: server umount lustre-OST0000 complete [10841.570891] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10841.604207] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10841.611793] Lustre: Skipped 1 previous similar message [10844.999312] Lustre: server umount lustre-MDT0000 complete [10848.978905] LustreError: 316855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787857118 with bad export cookie 10875767730076930143 [10848.989504] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10848.994123] LustreError: 316855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10849.420146] Lustre: server umount lustre-MDT0001 complete [10853.235932] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10855.045239] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 14:58:42 (1787857122) [10861.705914] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10862.236975] LustreError: 321147:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10862.256164] LustreError: 321147:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 3 previous similar messages [10862.346979] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10866.772427] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [10867.684836] LustreError: 321148:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10872.802652] LustreError: 321147:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10875.387602] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10875.597877] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10880.092800] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [10887.712496] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10893.169111] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10899.577762] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10899.996963] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10906.184786] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [10913.781363] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10920.129716] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10920.298204] Lustre: lustre-OST0001: new disk, initializing [10920.302305] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10920.309122] Lustre: 323980:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10920.421509] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10926.182540] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10926.189366] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10926.253482] LustreError: 324300:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10926.264401] LustreError: 324300:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10926.557805] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [10927.275784] LustreError: 324296:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10928.362452] LustreError: 324296:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10928.382845] LustreError: 324296:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10930.466547] LustreError: 324296:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10930.481712] LustreError: 324296:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10934.564125] LustreError: 324300:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10934.577813] LustreError: 324300:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [10936.436559] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10936.661916] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10942.579978] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8ee8f503c000.ost_server_uuid 50 [10942.885881] LustreError: 324300:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10942.899032] LustreError: 324300:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 16 previous similar messages [10944.399441] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8ee8f503c000.ost_server_uuid in FULL state after 0 sec [10951.151630] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8ee8f503c000.ost_server_uuid 50 [10952.826546] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8ee8f503c000.ost_server_uuid in FULL state after 0 sec [10954.358257] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10970.082765] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10970.097964] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10970.120248] Lustre: Skipped 1 previous similar message [10970.133363] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10974.720469] Lustre: server umount lustre-OST0001 complete [10977.761747] LustreError: 322969:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10977.782192] LustreError: 322969:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [10982.883488] LustreError: 322970:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10982.885275] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10982.899135] LustreError: 322970:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 2 previous similar messages [10982.911812] Lustre: Skipped 1 previous similar message [10985.786365] Lustre: server umount lustre-OST0000 complete [10993.125968] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10993.132816] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10993.149088] Lustre: Skipped 2 previous similar messages [10993.176595] Lustre: Skipped 7 previous similar messages [10995.595421] Lustre: server umount lustre-MDT0000 complete [10998.240467] LustreError: 321142:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10998.257684] LustreError: 321142:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [10998.283501] LustreError: 321129:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787857268 with bad export cookie 10875767730076931032 [10998.289359] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10998.296691] LustreError: 321129:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10998.585781] Lustre: server umount lustre-MDT0001 complete [11007.038715] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [11009.277576] Key type lgssc unregistered [11009.575614] LNet: 326686:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11009.584364] LNetError: 326686:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11009.602409] LNet: Removed LNI 192.168.206.150@tcp [11010.313418] Key type .llcrypt unregistered [11010.316819] Key type ._llcrypt unregistered [11032.622358] Key type ._llcrypt registered [11032.625754] Key type .llcrypt registered [11032.752029] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [11033.712241] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11033.776811] alg: No test for adler32 (adler32-zlib) [11034.971325] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [11035.234622] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [11036.888402] Key type lgssc registered [11037.938086] Lustre: Echo OBD driver; http://www.lustre.org/ [11045.466560] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11045.487919] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11046.977355] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11052.086434] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [11056.114052] LustreError: 328106:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11056.144408] LustreError: 328106:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 2 previous similar messages [11059.977627] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11061.216141] LustreError: 328107:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11066.339500] LustreError: 328106:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11067.006069] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11067.491594] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11072.288703] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [11081.174166] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11081.458930] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11087.198451] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [11091.684989] LustreError: 329787:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11091.707444] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11094.593702] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11098.092021] LustreError: 330364:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11098.113333] LustreError: 330364:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 3 previous similar messages [11101.615704] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11101.820553] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11107.329988] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11108.441652] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [11116.306185] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11124.386471] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11124.548349] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11230.181577] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11230.187062] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11230.195162] LustreError: Skipped 1 previous similar message [11230.200587] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11230.210291] Lustre: Skipped 1 previous similar message [11230.227986] Lustre: Skipped 1 previous similar message [11233.757346] Lustre: server umount lustre-OST0001 complete [11235.308142] LustreError: 330364:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11235.323169] LustreError: 330364:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [11240.419164] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11240.420969] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11240.433925] Lustre: Skipped 1 previous similar message [11240.454633] Lustre: Skipped 1 previous similar message [11245.553156] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11246.719344] Lustre: server umount lustre-OST0000 complete [11251.686719] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11251.696987] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11251.714049] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11251.717940] Lustre: Skipped 1 previous similar message [11255.777909] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11255.785441] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11255.799022] Lustre: Skipped 1 previous similar message [11257.105233] Lustre: server umount lustre-MDT0000 complete [11260.897258] LustreError: 329230:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11260.937273] LustreError: 329230:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 5 previous similar messages [11261.447073] LustreError: 331791:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787857531 with bad export cookie 3189722181082565980 [11261.457269] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11261.467623] LustreError: 331791:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11261.939772] Lustre: server umount lustre-MDT0001 complete [11275.695537] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [11279.971399] Key type lgssc unregistered [11280.330344] LNet: 333055:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11280.346535] LNetError: 333055:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11280.377266] LNet: Removed LNI 192.168.206.150@tcp [11281.790721] Key type .llcrypt unregistered [11281.796098] Key type ._llcrypt unregistered [11331.982710] Key type ._llcrypt registered [11331.984964] Key type .llcrypt registered [11332.107510] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [11332.725257] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11332.823344] alg: No test for adler32 (adler32-zlib) [11334.245963] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [11334.545248] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [11336.327160] Key type lgssc registered [11337.712827] Lustre: Echo OBD driver; http://www.lustre.org/ [11348.163978] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11348.205088] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11349.680754] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11354.988520] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [11359.206297] LustreError: 335346:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11359.229511] LustreError: 335346:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 2 previous similar messages [11363.758504] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11364.148901] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11369.222319] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [11372.550700] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11379.658616] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11380.137058] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11387.571762] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [11389.415336] LustreError: 336837:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11394.561751] LustreError: 337519:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11394.584781] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11394.600242] LustreError: 337519:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [11396.769068] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11397.011732] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11402.255273] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11403.276932] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [11413.490730] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11419.697429] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11432.928780] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11432.947448] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11432.961954] Lustre: Skipped 3 previous similar messages [11438.053538] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11438.063372] Lustre: Skipped 3 previous similar messages [11438.668829] Lustre: server umount lustre-MDT0000 complete [11442.113280] LustreError: 337664:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787857712 with bad export cookie 11931915056157075345 [11442.121627] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11442.127542] LustreError: 337664:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11442.449345] Lustre: server umount lustre-MDT0001 complete [11456.865568] Lustre: server umount lustre-OST0000 complete [11460.063261] Lustre: 334614:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787857713/real 1787857713] req@ffff9647be2cf480 x1874704575037568/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787857729 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11460.101527] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11460.114671] Lustre: Skipped 3 previous similar messages [11461.034519] Lustre: server umount lustre-OST0001 complete [11478.535436] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [11488.404276] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11488.725499] LustreError: 340612:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11488.834461] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11493.703196] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [11493.873103] LustreError: 340613:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11503.195020] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11503.529558] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11507.872429] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [11510.844748] Lustre: 341752:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11517.769160] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11524.421162] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [11528.369858] LustreError: 342106:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11528.396752] LustreError: 342106:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [11533.443485] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11533.516475] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11533.828753] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11533.837224] Lustre: Skipped 1 previous similar message [11538.943202] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11540.244724] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [11547.725517] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11551.660863] Lustre: 343622:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11559.402803] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11559.404903] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11559.423265] Lustre: Skipped 3 previous similar messages [11559.435657] Lustre: Skipped 3 previous similar messages [11562.503825] Lustre: server umount lustre-MDT0000 complete [11564.522948] LustreError: 340612:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11564.552398] LustreError: 340612:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 5 previous similar messages [11566.065808] LustreError: 340593:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787857835 with bad export cookie 11931915056157077046 [11566.068174] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11566.075027] LustreError: 340593:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11566.454953] Lustre: server umount lustre-MDT0001 complete [11580.814088] Lustre: server umount lustre-OST0000 complete [11593.471708] Lustre: server umount lustre-OST0001 complete [11596.807867] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11598.204608] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11599.818912] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 15:11:07 (1787857867) [11624.260874] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [11633.120214] loop0: detected capacity change from 0 to 1099511627776 [11633.772085] SGI XFS with ACLs, security attributes, quota, no debug enabled [11633.785045] *** VALIDATE xfs *** [11633.807191] XFS (loop0): Mounting V5 Filesystem [11633.856418] XFS (loop0): Ending clean mount [11635.134380] loop1: detected capacity change from 0 to 18691697672192 [11753.912742] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_hostid [11764.940557] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [11805.875625] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11806.061334] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11806.098511] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11806.188905] Lustre: lustre-MDT0000: new disk, initializing [11806.295057] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11806.306369] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11810.349393] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [11820.701313] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11820.777988] Lustre: 348763:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11820.831120] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11820.838690] Lustre: Skipped 1 previous similar message [11820.902405] Lustre: lustre-MDT0001: new disk, initializing [11821.044217] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11821.065347] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11825.065234] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [11833.974473] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11839.303080] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11845.395785] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11845.590828] Lustre: lustre-OST0000: new disk, initializing [11845.593944] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11845.598683] Lustre: 350730:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11846.881017] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11846.906771] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11846.986119] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11850.977745] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [11859.690922] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11864.660826] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11864.893679] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11869.460639] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11869.686180] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11872.736232] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11872.742656] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11872.745891] LustreError: Skipped 1 previous similar message [11872.748455] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11872.778905] Lustre: Skipped 1 previous similar message [11877.551400] Lustre: server umount lustre-OST0000 complete [11882.464337] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11882.471777] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11882.499179] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11882.501747] Lustre: Skipped 1 previous similar message [11887.337707] Lustre: server umount lustre-MDT0000 complete [11888.108564] LustreError: 348774:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11888.133437] LustreError: 348774:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [11891.170468] LustreError: 349556:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787858161 with bad export cookie 11931915056157078600 [11891.176803] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11891.186358] LustreError: 349556:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11891.598249] Lustre: server umount lustre-MDT0001 complete [11899.407977] XFS (loop0): Unmounting Filesystem [11904.317176] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 15:16:11 (1787858171) [11911.405545] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11911.922952] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11911.929732] Lustre: Skipped 2 previous similar messages [11916.484864] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [11924.345431] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11929.202705] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [11937.333184] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11943.300736] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11949.727730] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11956.531116] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [11964.771927] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11979.756741] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11979.759699] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11979.761976] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11979.761985] Lustre: Skipped 1 previous similar message [11979.774799] Lustre: Skipped 2 previous similar messages [11983.074407] Lustre: server umount lustre-OST0000 complete [11992.823630] Lustre: server umount lustre-MDT0000 complete [11995.111568] LustreError: 353290:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11995.134225] LustreError: 353290:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 4 previous similar messages [11996.813078] LustreError: 353273:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787858266 with bad export cookie 11931915056157079482 [11996.819259] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11996.834626] LustreError: 353273:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11997.216928] Lustre: server umount lustre-MDT0001 complete [12007.671760] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [12010.437701] Key type lgssc unregistered [12010.761733] LNet: 356923:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12010.772858] LNetError: 356923:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12011.817714] LNet: Removed LNI 192.168.206.150@tcp [12012.762185] Key type .llcrypt unregistered [12012.763918] Key type ._llcrypt unregistered [12039.974592] Key type ._llcrypt registered [12039.976061] Key type .llcrypt registered [12040.105761] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [12040.975950] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12041.034840] alg: No test for adler32 (adler32-zlib) [12042.189141] Lustre: Lustre: Build Version: 2.17.57_87_ge30c52f [12042.495505] LNet: Added LNI 192.168.206.150@tcp [8/256/0/180] [12044.191155] Key type lgssc registered [12045.081867] Lustre: Echo OBD driver; http://www.lustre.org/ [12052.251960] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12052.275296] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12053.666921] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12057.859152] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [12063.213861] LustreError: 358489:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12063.244614] LustreError: 358489:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 2 previous similar messages [12066.093508] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12066.403125] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12070.533654] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [12077.865932] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12083.111354] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12089.755871] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12090.049481] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12096.304719] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [12104.186699] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12112.488724] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.50@tcp (stopping) [12112.865754] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12112.875805] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12113.889889] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12113.902071] Lustre: Skipped 1 previous similar message [12113.905174] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12117.561634] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.50@tcp (stopping) [12117.569292] Lustre: Skipped 1 previous similar message [12118.633530] Lustre: server umount lustre-MDT0001 complete [12119.013208] LustreError: 359216:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12119.037034] LustreError: 359216:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [12122.665624] LustreError: 360937:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.50@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12127.783766] LustreError: 358484:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.50@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12127.803315] LustreError: 358484:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 2 previous similar messages [12128.680374] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12128.820261] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12128.825840] Lustre: Found index 1 for lustre-MDT0001, updating log [12128.850122] Lustre: 358478:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12129.131295] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12133.186733] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [12134.384905] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12134.405587] LustreError: 362005:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12134.410589] LustreError: 362005:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12134.420290] LustreError: 358482:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12134.489442] LustreError: 362009:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.150@tcp: cfg command failed: rc = -17 [12134.496908] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12134.496908] [12143.183751] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.50@tcp (stopping) [12144.609697] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12144.613875] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12144.638476] Lustre: Skipped 1 previous similar message [12147.970399] Lustre: server umount lustre-MDT0001 complete [12148.264229] LustreError: 360450:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.50@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12158.500941] LustreError: 358486:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.50@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12158.512368] LustreError: 358486:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 5 previous similar messages [12159.329081] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12159.500178] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12159.504829] Lustre: Found index 1 for lustre-MDT0001, updating log [12159.575299] Lustre: 358477:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12159.997248] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12164.912953] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [12165.126689] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12166.646737] LustreError: 363093:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12166.654659] LustreError: 363093:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12166.662836] LustreError: 358482:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12166.669876] LustreError: 358482:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12166.746850] LustreError: 363104:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12166.757054] LustreError: 363104:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12166.774680] LustreError: 363104:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.150@tcp: cfg command failed: rc = -17 [12166.788580] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.206.150@tcp [12166.788580] [12470.849768] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.50@tcp (stopping) [12470.857057] Lustre: Skipped 2 previous similar messages [12472.290278] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12472.291699] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12476.419438] Lustre: server umount lustre-MDT0001 complete [12477.427440] LustreError: 359216:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12477.461688] LustreError: 359216:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 1 previous similar message [12488.488978] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12488.614088] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12488.618727] Lustre: Found index 1 for lustre-MDT0001, updating log [12488.681233] Lustre: 358479:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12489.072283] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12493.724926] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [12494.319663] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12495.867065] LustreError: 371601:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12495.874338] LustreError: 371601:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12495.883788] LustreError: 358482:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12495.891834] LustreError: 358482:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12495.964882] LustreError: 371606:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12495.976292] LustreError: 371606:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.150@tcp: cfg command failed: rc = -114 [12495.982698] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12495.982698] [12816.886422] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12816.892489] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12816.892610] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12816.899631] Lustre: Skipped 3 previous similar messages [12816.931260] Lustre: Skipped 5 previous similar messages [12821.893265] Lustre: server umount lustre-MDT0000 complete [12821.989314] LustreError: 360937:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12822.001176] LustreError: 360937:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 8 previous similar messages [12825.113500] LustreError: 359223:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787859095 with bad export cookie 11441351212915040246 [12825.115337] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12825.126074] LustreError: 359223:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12827.107435] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12835.238979] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12835.243710] Lustre: Skipped 2 previous similar messages [12839.392164] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12839.702887] Lustre: server umount lustre-MDT0001 complete [12843.307534] Lustre: server umount lustre-OST0000 complete [12852.421234] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_hostid [12862.178465] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [12899.917460] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 15:32:47 (1787859167) [12907.116484] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12907.397473] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12907.417967] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12907.492504] Lustre: lustre-MDT0000: new disk, initializing [12907.567191] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12907.583581] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12912.112931] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [12924.252186] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12924.359214] Lustre: 382516:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12924.415115] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12924.419277] Lustre: Skipped 1 previous similar message [12924.503733] Lustre: lustre-MDT0001: new disk, initializing [12924.693619] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12924.778385] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12924.790081] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12929.105516] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [12939.705278] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12945.284712] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12954.181336] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12954.520845] Lustre: lustre-OST0000: new disk, initializing [12954.525924] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12954.535915] Lustre: 384486:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12954.693425] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12964.680554] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_default_debug -1 all [12964.983478] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12965.001013] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12965.170186] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12978.016524] Lustre: DEBUG MARKER: oleg650-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12985.827911] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12985.841702] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12985.855020] Lustre: Skipped 2 previous similar messages [12985.862193] Lustre: Skipped 1 previous similar message [12987.792316] Lustre: server umount lustre-MDT0000 complete [12990.956818] LustreError: 382529:0:(ldlm_lib.c:1200:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12990.986363] LustreError: 382529:0:(ldlm_lib.c:1200:target_handle_connect()) Skipped 5 previous similar messages [12991.818513] LustreError: 382509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787859261 with bad export cookie 11441351212915042192 [12991.822343] LustreError: MGC192.168.206.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12991.834298] LustreError: 382509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12992.197593] Lustre: server umount lustre-MDT0001 complete [13006.647346] Lustre: server umount lustre-OST0000 complete [13040.654356] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing set_hostid [13052.239499] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing load_modules_local [13087.072843] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12846 sec ========= 15:35:54 (1787859354) [13089.022260] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 15:35:56 (1787859356) === [13092.585737] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 15:35:59 (1787859359) === [13138.388972] Lustre: DEBUG MARKER: oleg650-server.virtnet: executing unload_modules_local [13140.960516] Key type lgssc unregistered [13141.311428] LNet: 390857:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13141.332786] LNetError: 390857:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13141.367954] LNet: Removed LNI 192.168.206.150@tcp [13142.291217] Key type .llcrypt unregistered [13142.296441] Key type ._llcrypt unregistered