[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 515795609 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001009] APIC: Switch to symmetric I/O mode setup [ 0.002224] x2apic enabled [ 0.003006] Switched APIC routing to physical x2apic. [ 0.004007] kvm-guest: setup PV IPIs [ 0.005924] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.006000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.006021] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.007009] pid_max: default: 32768 minimum: 301 [ 0.008120] LSM: Security Framework initializing [ 0.009039] Yama: becoming mindful. [ 0.010024] SELinux: Initializing. [ 0.010728] *** VALIDATE selinux *** [ 0.018404] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.023352] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.024137] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.025109] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.026096] *** VALIDATE tmpfs *** [ 0.028501] *** VALIDATE proc *** [ 0.029231] *** VALIDATE cgroup *** [ 0.030009] *** VALIDATE cgroup2 *** [ 0.032131] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.033117] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.034008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.035021] Spectre V2 : User space: Vulnerable [ 0.036004] Speculative Store Bypass: Vulnerable [ 0.038622] debug: unmapping init [mem 0xffffffffa9059000-0xffffffffa9060fff] [ 0.040150] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.041752] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.042031] ... version: 2 [ 0.043012] ... bit width: 48 [ 0.044010] ... generic registers: 4 [ 0.044911] ... value mask: 0000ffffffffffff [ 0.045011] ... max period: 00007fffffffffff [ 0.046010] ... fixed-purpose events: 3 [ 0.046735] ... event mask: 000000070000000f [ 0.047300] rcu: Hierarchical SRCU implementation. [ 0.049231] smp: Bringing up secondary CPUs ... [ 0.050442] x86: Booting SMP configuration: [ 0.051027] .... node #0, CPUs: #1 #2 #3 [ 0.058084] smp: Brought up 1 node, 4 CPUs [ 0.060036] smpboot: Max logical packages: 1 [ 0.061013] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.132854] node 0 deferred pages initialised in 69ms [ 0.135094] devtmpfs: initialized [ 0.135957] x86/mm: Memory block size: 128MB [ 0.137147] gcov: version magic: 0x41383552 [ 0.139188] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.141078] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.142302] pinctrl core: initialized pinctrl subsystem [ 0.144134] [ 0.144407] ************************************************************* [ 0.145008] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.146007] ** ** [ 0.148007] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.150016] ** ** [ 0.152018] ** This means that this kernel is built to expose internal ** [ 0.154010] ** IOMMU data structures, which may compromise security on ** [ 0.155009] ** your system. ** [ 0.157008] ** ** [ 0.159013] ** If you see this message and you are not debugging the ** [ 0.161013] ** kernel, report this immediately to your vendor! ** [ 0.163012] ** ** [ 0.165011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.167008] ************************************************************* [ 0.168534] NET: Registered protocol family 16 [ 0.170291] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.171043] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.173035] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.176010] cpuidle: using governor menu [ 0.177607] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.179320] PCI: Using configuration type 1 for base access [ 0.180000] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.185069] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.187030] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.189396] cryptd: max_cpu_qlen set to 1000 [ 0.191250] ACPI: Added _OSI(Module Device) [ 0.192009] ACPI: Added _OSI(Processor Device) [ 0.194009] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.195009] ACPI: Added _OSI(Processor Aggregator Device) [ 0.198213] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.203400] ACPI: Interpreter enabled [ 0.204045] ACPI: PM: (supports S0 S3 S4 S5) [ 0.205010] ACPI: Using IOAPIC for interrupt routing [ 0.206119] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.210490] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.219277] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.221047] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.223016] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.225095] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.227998] acpiphp: Slot [2] registered [ 0.228892] acpiphp: Slot [5] registered [ 0.230080] acpiphp: Slot [6] registered [ 0.231101] acpiphp: Slot [7] registered [ 0.232053] acpiphp: Slot [8] registered [ 0.234142] acpiphp: Slot [9] registered [ 0.235103] acpiphp: Slot [10] registered [ 0.237149] acpiphp: Slot [3] registered [ 0.238101] acpiphp: Slot [4] registered [ 0.239103] acpiphp: Slot [11] registered [ 0.241112] acpiphp: Slot [12] registered [ 0.242102] acpiphp: Slot [13] registered [ 0.243109] acpiphp: Slot [14] registered [ 0.244121] acpiphp: Slot [15] registered [ 0.246137] acpiphp: Slot [16] registered [ 0.247106] acpiphp: Slot [17] registered [ 0.248101] acpiphp: Slot [18] registered [ 0.250121] acpiphp: Slot [19] registered [ 0.252191] acpiphp: Slot [20] registered [ 0.253088] acpiphp: Slot [21] registered [ 0.254113] acpiphp: Slot [22] registered [ 0.255080] acpiphp: Slot [23] registered [ 0.256087] acpiphp: Slot [24] registered [ 0.257117] acpiphp: Slot [25] registered [ 0.259113] acpiphp: Slot [26] registered [ 0.260058] acpiphp: Slot [27] registered [ 0.260912] acpiphp: Slot [28] registered [ 0.262079] acpiphp: Slot [29] registered [ 0.263038] acpiphp: Slot [30] registered [ 0.263916] acpiphp: Slot [31] registered [ 0.264055] PCI host bridge to bus 0000:00 [ 0.266020] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.267016] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.269018] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.271028] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.273020] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.274015] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.276175] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.280060] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.283389] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.292806] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.297059] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.300024] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.304024] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.306019] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.309571] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.312875] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.315153] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.318798] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.322011] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.333014] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.338013] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.347000] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.353020] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.366021] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.385020] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.392994] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.399016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.405015] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.419027] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.426452] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.432018] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.437014] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.448013] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.459556] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.465016] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.470017] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.483018] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.492110] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.497017] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.503018] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.515021] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.524126] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.528012] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.533018] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.543016] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.552134] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.555409] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.558257] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.560347] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.562206] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.568155] iommu: Default domain type: Passthrough [ 0.570317] SCSI subsystem initialized [ 0.571133] ACPI: bus type USB registered [ 0.572087] usbcore: registered new interface driver usbfs [ 0.574091] usbcore: registered new interface driver hub [ 0.575062] usbcore: registered new device driver usb [ 0.577124] pps_core: LinuxPPS API ver. 1 registered [ 0.578007] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.580077] PTP clock support registered [ 0.582459] EDAC MC: Ver: 3.0.0 [ 0.583000] PCI: Using ACPI for IRQ routing [ 0.584793] NetLabel: Initializing [ 0.585006] NetLabel: domain hash size = 128 [ 0.585941] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.588085] NetLabel: unlabeled traffic allowed by default [ 0.591004] vgaarb: loaded [ 0.592479] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.594011] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.601000] clocksource: Switched to clocksource kvm-clock [ 0.705330] VFS: Disk quotas dquot_6.6.0 [ 0.707021] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.709634] *** VALIDATE ramfs *** [ 0.710993] *** VALIDATE hugetlbfs *** [ 0.712698] pnp: PnP ACPI init [ 0.715871] pnp: PnP ACPI: found 6 devices [ 0.732670] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.736358] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.738635] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.740920] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.743581] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.746380] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.749541] NET: Registered protocol family 2 [ 0.752964] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.758588] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.763865] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.770345] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.773862] TCP: Hash tables configured (established 65536 bind 65536) [ 0.777254] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.781366] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.784746] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.788818] NET: Registered protocol family 1 [ 0.792207] RPC: Registered named UNIX socket transport module. [ 0.794482] RPC: Registered udp transport module. [ 0.796351] RPC: Registered tcp transport module. [ 0.798156] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.800589] NET: Registered protocol family 44 [ 0.802290] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.804532] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.806519] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.808825] PCI: CLS 0 bytes, default 64 [ 0.810475] Unpacking initramfs... [ 2.202427] debug: unmapping init [mem 0xffff947c7cc54000-0xffff947c7ffbffff] [ 2.206612] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.209100] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.212303] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.727489] Initialise system trusted keyrings [ 2.729424] Key type blacklist registered [ 2.732157] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.742366] zbud: loaded [ 2.745745] *** VALIDATE nfs *** [ 2.747140] *** VALIDATE nfs4 *** [ 2.748977] pstore: using deflate compression [ 2.752712] Platform Keyring initialized [ 2.858743] NET: Registered protocol family 38 [ 2.860624] Key type asymmetric registered [ 2.862380] Asymmetric key parser 'x509' registered [ 2.864652] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.868131] io scheduler mq-deadline registered [ 2.869894] io scheduler kyber registered [ 2.871909] io scheduler bfq registered [ 2.873973] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.877085] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.879655] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.882647] ACPI: Power Button [PWRF] [ 2.888305] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.895189] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.911386] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.919589] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.937698] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.966220] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.996954] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.002334] Non-volatile memory driver v1.3 [ 3.004110] Linux agpgart interface v0.103 [ 3.037336] virtio_blk virtio1: [vda] 146696 512-byte logical blocks (75.1 MB/71.6 MiB) [ 3.040108] vda: detected capacity change from 0 to 75108352 [ 3.055166] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.058406] vdb: detected capacity change from 0 to 1073741824 [ 3.078182] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.080481] vdc: detected capacity change from 0 to 2621440000 [ 3.096251] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.099455] vdd: detected capacity change from 0 to 2621440000 [ 3.114539] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.116445] vde: detected capacity change from 0 to 4294967296 [ 3.129688] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.131468] vdf: detected capacity change from 0 to 4294967296 [ 3.140334] libphy: Fixed MDIO Bus: probed [ 3.148976] usbcore: registered new interface driver usbserial_generic [ 3.150904] usbserial: USB Serial support registered for generic [ 3.153082] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.156705] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.159062] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.161495] mousedev: PS/2 mouse device common for all mice [ 3.165351] rtc_cmos 00:05: RTC can wake from S4 [ 3.168338] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.168692] rtc_cmos 00:05: registered as rtc0 [ 3.173764] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.175941] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.178068] intel_pstate: CPU model not supported [ 3.183275] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.188083] hid: raw HID events driver (C) Jiri Kosina [ 3.189694] usbcore: registered new interface driver usbhid [ 3.191234] usbhid: USB HID core driver [ 3.192918] drop_monitor: Initializing network drop monitor service [ 3.195476] Initializing XFRM netlink socket [ 3.197222] NET: Registered protocol family 10 [ 3.199880] Segment Routing with IPv6 [ 3.201116] NET: Registered protocol family 17 [ 3.202672] mpls_gso: MPLS GSO support [ 3.208596] RAS: Correctable Errors collector initialized. [ 3.210715] AVX version of gcm_enc/dec engaged. [ 3.212442] AES CTR mode by8 optimization enabled [ 3.288210] sched_clock: Marking stable (3288186915, 0)->(4095271885, -807084970) [ 3.290753] registered taskstats version 1 [ 3.292048] Loading compiled-in X.509 certificates [ 3.293366] zswap: loaded using pool lzo/zbud [ 3.315027] Key type big_key registered [ 3.325175] Key type encrypted registered [ 3.326191] ima: No TPM chip found, activating TPM-bypass! [ 3.327713] ima: Allocated hash algorithm: sha1 [ 3.329105] ima: No architecture policies found [ 3.330515] evm: Initialising EVM extended attributes: [ 3.332463] evm: security.selinux [ 3.333659] evm: security.ima [ 3.334761] evm: security.capability [ 3.336143] evm: HMAC attrs: 0x1 [ 3.338331] rtc_cmos 00:05: setting system clock to 2026-09-13 11:02:47 UTC (1789297367) [ 3.342677] debug: unmapping init [mem 0xffffffffaa003000-0xffffffffaa1fffff] [ 3.345370] debug: unmapping init [mem 0xffffffffa8d82000-0xffffffffa9058fff] [ 3.357098] Write protecting the kernel read-only data: 28672k [ 3.360845] debug: unmapping init [mem 0xffffffffa7403000-0xffffffffa75fffff] [ 3.364495] debug: unmapping init [mem 0xffffffffa7d14000-0xffffffffa7dfffff] [ 3.402589] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.411486] systemd[1]: Detected virtualization kvm. [ 3.413733] systemd[1]: Detected architecture x86-64. [ 3.415887] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.443342] systemd[1]: No hostname configured. [ 3.445254] systemd[1]: Set hostname to . [ 3.447412] random: systemd: uninitialized urandom read (16 bytes read) [ 3.450328] systemd[1]: Initializing machine ID from random generator. [ 3.600217] random: systemd: uninitialized urandom read (16 bytes read) [ 3.602374] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 3.607074] random: systemd: uninitialized urandom read (16 bytes read) [ 3.609561] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 3.613865] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Initrd Root Device. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Slices. [ OK ] Reached target Swap. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... Starting Create Volatile Files and Directories... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... Starting Journal Service... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.212395] device-mapper: uevent: version 1.0.3 [ 4.213828] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 4.927375] virtio_net virtio0 ens2: renamed from eth0 [ 4.941709] random: fast init done [ 5.065324] scsi host0: ata_piix [ 5.078580] scsi host1: ata_piix [ 5.080233] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.082730] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.576162] dracut-initqueue[589]: RTNETLINK answers: File exists [ 9.846972] random: crng init done [ 9.848685] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.275484] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped Apply Kernel Variables. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.312285] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.510345] SELinux: Disabled at runtime. [ 11.566995] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.575691] systemd[1]: Detected virtualization kvm. [ 11.577084] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.039412] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.042683] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.047522] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.051329] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.055661] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.064232] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.069258] systemd[1]: Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-serial\x2dgetty.slice. Mounting POSIX Message Queue File System... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice User and Session Slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Slices. [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Apply Kernel Variables... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target rpc_pipefs.target. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Mounting Huge Pages File System... Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-getty.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Process Core Dump Socket. Mounting Kernel Debug File System... Activating swap /dev/disk/by-label/SWAP... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Reached target Paths. [ OK ] Stopped targ[ 12.222541] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS et Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Huge Pages File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 12.508386] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 12.757588] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 12.838857] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 12.963980] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 12.982295] EDAC sbridge: Ver: 1.1.2 [ 14.334787] Key type dns_resolver registered [ 14.628671] NFS: Registering the id_resolver key type [ 14.630479] Key type id_resolver registered [ 14.632112] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Hostname Service... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting Network Manager Wait Online... [ OK ] Started Login Service. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg155-server login: [ 39.732731] libcfs: loading out-of-tree module taints kernel. [ 39.752739] Key type ._llcrypt registered [ 39.753896] Key type .llcrypt registered [ 39.799407] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_hostid [ 48.087330] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 48.711121] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 48.719283] alg: No test for adler32 (adler32-zlib) [ 49.735405] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [ 50.097287] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [ 51.719186] Key type lgssc registered [ 52.375415] Lustre: Echo OBD driver; http://www.lustre.org/ [ 64.528734] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 111.019251] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 125.787911] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 125.850921] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 127.218222] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 127.258863] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 127.413527] Lustre: lustre-MDT0000: new disk, initializing [ 127.546497] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 127.561380] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 132.610976] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 146.713176] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 146.859834] Lustre: 6488:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 146.882953] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 146.891634] Lustre: Skipped 1 previous similar message [ 146.958459] Lustre: lustre-MDT0001: new disk, initializing [ 147.066312] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 147.125645] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 147.141119] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 151.960155] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 157.273880] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 168.124563] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 168.388210] Lustre: lustre-OST0000: new disk, initializing [ 168.392236] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 168.402379] Lustre: 8428:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 168.462064] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 169.623347] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 169.629199] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 169.761047] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 176.150514] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 185.347300] hrtimer: interrupt took 2347697 ns [ 192.878657] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 193.128304] Lustre: lustre-OST0001: new disk, initializing [ 193.131927] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 193.139316] Lustre: 9501:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 193.235526] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 199.267176] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 199.289176] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 199.405183] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 201.905427] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 214.616691] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 222.224806] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 230.635765] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing check_logdir /tmp/testlogs/ [ 236.954988] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing yml_node [ 241.825824] Lustre: DEBUG MARKER: Client: 2.17.57.108 [ 244.398224] Lustre: DEBUG MARKER: MDS: 2.17.57.108 [ 247.595909] Lustre: DEBUG MARKER: OSS: 2.17.57.108 [ 249.422299] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sun Sep 13 07:06:52 EDT 2026 [ 267.331645] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 269.154546] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 286.688965] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 286.692221] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 286.703641] Lustre: Skipped 2 previous similar messages [ 286.732681] Lustre: Skipped 3 previous similar messages [ 291.652519] Lustre: server umount lustre-MDT0000 complete [ 295.809990] LustreError: 6482:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789297659 with bad export cookie 13210524990607877778 [ 295.813457] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 295.816095] LustreError: 6482:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 296.929642] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 296.930267] LustreError: 6496:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 296.930530] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 296.939115] Lustre: Skipped 2 previous similar messages [ 296.945484] LustreError: 6496:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 300.959933] LustreError: 6495:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 300.965305] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 300.976063] LustreError: 6495:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 300.998867] Lustre: Skipped 2 previous similar messages [ 305.057158] LustreError: 7751:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 305.057692] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 305.072951] LustreError: 7751:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 305.088942] Lustre: Skipped 1 previous similar message [ 310.239773] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 310.241262] LustreError: 6494:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 310.241335] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 310.299402] LustreError: 6494:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 310.718584] Lustre: server umount lustre-MDT0001 complete [ 314.868205] Lustre: server umount lustre-OST0000 complete [ 318.546802] Lustre: server umount lustre-OST0001 complete [ 325.888151] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_hostid [ 338.930586] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 380.357236] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 380.602714] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 380.664619] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 380.792437] Lustre: lustre-MDT0000: new disk, initializing [ 380.914904] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 380.952497] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 386.418178] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 398.701329] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 398.849322] Lustre: 15057:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 398.928707] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 398.932055] Lustre: Skipped 1 previous similar message [ 398.988400] Lustre: lustre-MDT0001: new disk, initializing [ 399.083310] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 399.126832] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 399.158827] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 404.804340] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 415.100357] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 419.805911] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 426.285680] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 426.606898] Lustre: lustre-OST0000: new disk, initializing [ 426.610990] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 426.617264] Lustre: 17028:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 426.743305] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 428.433148] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 428.451453] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 428.653046] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 433.376359] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 442.597491] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 447.043124] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 447.343095] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 451.919272] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 452.150661] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 454.116241] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 454.122471] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 454.123470] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 454.125707] Lustre: Skipped 1 previous similar message [ 454.128663] Lustre: Skipped 1 previous similar message [ 460.318644] Lustre: server umount lustre-OST0000 complete [ 464.353150] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 464.370929] Lustre: Skipped 1 previous similar message [ 470.471355] Lustre: server umount lustre-MDT0000 complete [ 473.707864] LustreError: 15049:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789297837 with bad export cookie 13210524990607879304 [ 473.709690] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 473.721990] LustreError: 15049:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 474.001431] Lustre: server umount lustre-MDT0001 complete [ 478.476603] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 07:10:41 (1789297841) [ 485.550184] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 486.180781] LustreError: 19293:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 486.240585] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 490.579208] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 496.614476] LustreError: 19294:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 496.629332] LustreError: 19294:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 499.775700] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 505.229650] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 513.039312] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 519.232649] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 526.003450] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 533.440191] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 541.976791] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 548.838232] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 555.489836] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 555.500428] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 555.518487] Lustre: Skipped 2 previous similar messages [ 555.524245] Lustre: Skipped 7 previous similar messages [ 561.683222] Lustre: server umount lustre-MDT0000 complete [ 564.786534] LustreError: 19273:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789297928 with bad export cookie 13210524990607880193 [ 564.787738] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 564.798723] LustreError: 19273:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 565.361579] Lustre: server umount lustre-MDT0001 complete [ 578.406043] Lustre: server umount lustre-OST0000 complete [ 599.847881] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 609.282569] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 609.695575] LustreError: 23673:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 609.768624] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 609.777377] Lustre: Skipped 2 previous similar messages [ 614.268738] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 623.119312] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 629.744359] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 633.446201] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 641.030060] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 647.497144] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 657.238569] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 657.426457] Lustre: lustre-OST0001: new disk, initializing [ 657.429908] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 657.435631] Lustre: 25989:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 659.173646] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 659.194868] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 659.354185] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 664.219200] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 674.750824] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 678.707604] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 690.151120] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 690.163365] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 690.164660] Lustre: Skipped 2 previous similar messages [ 690.164986] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 690.164989] LustreError: Skipped 1 previous similar message [ 690.189562] Lustre: Skipped 7 previous similar messages [ 695.647141] Lustre: server umount lustre-MDT0000 complete [ 699.511442] LustreError: 23655:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789298063 with bad export cookie 13210524990607881040 [ 699.525941] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 699.550389] LustreError: 23655:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 700.201465] Lustre: server umount lustre-MDT0001 complete [ 715.070645] Lustre: server umount lustre-OST0000 complete [ 716.833857] Lustre: 3622:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789298064/real 1789298064] req@ffff947bc4e68e00 x1876214325430144/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789298080 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 716.860932] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 716.876942] Lustre: Skipped 1 previous similar message [ 719.661445] Lustre: server umount lustre-OST0001 complete [ 737.603782] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 749.908138] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 750.178469] LustreError: 29009:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 750.190352] LustreError: 29009:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 750.274249] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 750.280110] Lustre: Skipped 3 previous similar messages [ 754.578213] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 763.017101] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 768.442896] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 771.504257] Lustre: 30156:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 777.874233] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 784.369082] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 793.041350] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 800.024618] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 807.989618] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 812.351940] Lustre: 32035:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 819.680481] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 819.690219] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 819.708122] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 819.713242] Lustre: Skipped 4 previous similar messages [ 825.179040] Lustre: server umount lustre-MDT0000 complete [ 826.342305] LustreError: 29009:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 826.363134] LustreError: 29009:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 829.140223] LustreError: 28989:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789298193 with bad export cookie 13210524990607882335 [ 829.161992] LustreError: 28989:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 829.166074] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 829.589441] Lustre: server umount lustre-MDT0001 complete [ 843.095667] Lustre: server umount lustre-OST0000 complete [ 857.585755] Lustre: server umount lustre-OST0001 complete [ 866.996696] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 07:17:09 (1789298229) [ 893.204911] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 903.378416] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 909.031135] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 917.384594] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 922.100032] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 925.018886] Lustre: 35987:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 931.710817] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 938.700771] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 947.306581] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 954.539420] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 963.127432] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 966.466657] Lustre: 37868:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 974.316791] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 974.322235] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 974.333092] Lustre: Skipped 5 previous similar messages [ 974.341453] Lustre: Skipped 6 previous similar messages [ 974.343697] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 977.969564] Lustre: server umount lustre-MDT0000 complete [ 979.437812] LustreError: 34833:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 979.456055] LustreError: 34833:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 981.649873] LustreError: 34818:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789298345 with bad export cookie 13210524990607883441 [ 981.665901] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 981.668624] LustreError: 34818:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 982.041287] Lustre: server umount lustre-MDT0001 complete [ 995.190552] Lustre: server umount lustre-OST0000 complete [ 1009.453633] Lustre: server umount lustre-OST0001 complete [ 1017.659569] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 07:19:40 (1789298380) [ 1033.244972] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 1042.346165] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1042.739975] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1042.751443] Lustre: Skipped 7 previous similar messages [ 1047.278946] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 1055.694241] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1061.187542] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 1064.384619] Lustre: 41160:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1071.727665] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1078.853229] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 1086.879762] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1094.795502] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 1102.161682] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1105.468839] Lustre: 43039:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1106.175219] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1106.186861] Lustre: Skipped 1 previous similar message [ 1108.962636] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1108.975610] Lustre: Skipped 3 previous similar messages [ 1115.181694] Lustre: server umount lustre-MDT0000 complete [ 1118.849220] LustreError: 39991:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789298483 with bad export cookie 13210524990607884547 [ 1118.861233] LustreError: 39991:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1118.870019] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1125.297894] Lustre: server umount lustre-MDT0001 complete [ 1135.534271] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1144.737732] LustreError: 3619:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff947cc5cb3800 x1876214325579776/t0(0) o250->MGC192.168.201.155@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1150.742994] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 1153.453090] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1160.716435] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1166.189473] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 1166.847148] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1166.865808] Lustre: Skipped 1 previous similar message [ 1175.416873] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1177.883401] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1184.716784] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1186.088038] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1191.905113] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1196.914316] Lustre: server umount lustre-MDT0000 complete [ 1201.855458] LustreError: 39990:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789298566 with bad export cookie 13210524990607885646 [ 1201.856607] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1201.874778] LustreError: 39990:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1202.711946] Lustre: server umount lustre-MDT0001 complete [ 1217.732363] Lustre: server umount lustre-OST0000 complete [ 1218.532947] Lustre: 3621:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789298566/real 1789298566] req@ffff947cf6e1c380 x1876214325616000/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789298582 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1221.762943] Lustre: server umount lustre-OST0001 complete [ 1231.781716] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 07:23:14 (1789298594) [ 1247.133595] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 1257.584081] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1257.842206] LustreError: 47647:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1257.855159] LustreError: 47647:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 25 previous similar messages [ 1262.103954] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 1270.002360] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1275.280174] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 1278.275670] Lustre: 48801:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1285.121243] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1293.001962] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 1303.747842] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1311.093478] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 1318.978227] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1324.725662] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1337.773212] Lustre: server umount lustre-MDT0000 complete [ 1341.853952] LustreError: 47628:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789298706 with bad export cookie 13210524990607886465 [ 1341.856891] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1341.864493] LustreError: 47628:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1342.414754] Lustre: server umount lustre-MDT0001 complete [ 1357.602983] Lustre: server umount lustre-OST0000 complete [ 1361.185219] Lustre: 3622:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789298709/real 1789298709] req@ffff947cc5cb1180 x1876214325675136/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789298725 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1361.954952] Lustre: server umount lustre-OST0001 complete [ 1372.471767] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 07:25:35 (1789298735) [ 1375.202740] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1377.092403] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 07:25:39 (1789298739) [ 1387.081785] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_hostid [ 1396.291349] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 1418.597317] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1418.907496] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1418.997809] Lustre: lustre-MDT0000: new disk, initializing [ 1419.151348] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1424.207275] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 1434.229264] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1441.926338] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1442.355654] Lustre: lustre-OST0000: new disk, initializing [ 1442.359952] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1442.364967] Lustre: Skipped 1 previous similar message [ 1442.374280] Lustre: 54876:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1444.212899] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1444.237255] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1444.276027] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1449.248989] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 1460.095678] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1462.474899] Lustre: DEBUG MARKER: create test files [ 1495.539929] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1495.555418] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1495.587951] Lustre: Skipped 13 previous similar messages [ 1495.601069] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1495.607273] Lustre: Skipped 19 previous similar messages [ 1500.399458] Lustre: server umount lustre-OST0000 complete [ 1504.614757] Lustre: server umount lustre-MDT0000 complete [ 1516.899647] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [ 1519.919889] Key type lgssc unregistered [ 1520.311535] LNet: 56507:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1520.325623] LNetError: 56507:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1520.357594] LNet: Removed LNI 192.168.201.155@tcp [ 1521.834269] Key type .llcrypt unregistered [ 1521.837584] Key type ._llcrypt unregistered [ 1553.928937] Key type ._llcrypt registered [ 1553.930653] Key type .llcrypt registered [ 1554.060157] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 1555.431183] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1555.453789] alg: No test for adler32 (adler32-zlib) [ 1556.759349] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [ 1556.889125] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [ 1558.511846] Key type lgssc registered [ 1561.019806] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1569.368903] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1569.414873] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1571.097486] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1577.592438] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 1588.419635] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1597.318242] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1597.728540] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1603.354983] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 1611.923324] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1616.046112] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:99 to 0x240000400:129) [ 1616.079562] Lustre: 59242:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1616.084174] Lustre: 59242:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1616.092994] Lustre: 59242:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1616.100193] Lustre: 59242:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1616.105121] Lustre: 59242:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1616.109520] Lustre: 59242:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1622.278838] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1627.765734] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1636.840535] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1636.848071] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1636.859934] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1641.532800] Lustre: server umount lustre-OST0000 complete [ 1645.560946] Lustre: server umount lustre-MDT0000 complete [ 1657.839953] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [ 1660.583190] Key type lgssc unregistered [ 1661.044935] LNet: 60854:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1661.064439] LNetError: 60854:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1661.081258] LNet: Removed LNI 192.168.201.155@tcp [ 1662.080240] Key type .llcrypt unregistered [ 1662.085058] Key type ._llcrypt unregistered [ 1689.353751] Key type ._llcrypt registered [ 1689.355649] Key type .llcrypt registered [ 1689.472533] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 1690.869621] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1690.948528] alg: No test for adler32 (adler32-zlib) [ 1691.941713] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [ 1692.110880] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [ 1693.743260] Key type lgssc registered [ 1694.788840] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1704.733568] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1704.760890] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1706.487650] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1710.874752] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 1719.841252] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1727.586787] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1728.038780] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1731.128042] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1731.189378] Lustre: 63664:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1731.197067] Lustre: 63664:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1731.199064] Lustre: 63664:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1731.201116] Lustre: 63664:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1731.203158] Lustre: 63664:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1731.206089] Lustre: 63664:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1735.708305] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 1744.439321] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1749.682345] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1761.788833] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1761.798938] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1761.836199] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1764.342795] Lustre: server umount lustre-OST0000 complete [ 1770.397765] Lustre: server umount lustre-MDT0000 complete [ 1784.709830] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [ 1786.820542] Key type lgssc unregistered [ 1787.006872] LNet: 65224:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1787.014360] LNetError: 65224:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1788.071289] LNet: Removed LNI 192.168.201.155@tcp [ 1789.024351] Key type .llcrypt unregistered [ 1789.030310] Key type ._llcrypt unregistered [ 1808.068862] Key type ._llcrypt registered [ 1808.071126] Key type .llcrypt registered [ 1808.173733] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_hostid [ 1830.677626] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 1832.569487] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1832.685418] alg: No test for adler32 (adler32-zlib) [ 1833.924978] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [ 1834.432647] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [ 1836.223436] Key type lgssc registered [ 1837.547083] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1872.551377] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1872.566384] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1873.916059] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1873.961977] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1874.076962] Lustre: lustre-MDT0000: new disk, initializing [ 1874.209267] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1874.239954] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1878.691990] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 1890.764239] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1890.909893] Lustre: 68381:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1890.975569] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1890.980578] Lustre: Skipped 1 previous similar message [ 1891.076308] Lustre: lustre-MDT0001: new disk, initializing [ 1891.267269] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1891.347642] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1891.371737] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1898.344948] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 1908.135800] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1913.796333] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1920.855133] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1921.164201] Lustre: lustre-OST0000: new disk, initializing [ 1921.172675] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1921.177292] Lustre: 70351:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1921.291960] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1927.897388] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 1930.308176] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1930.321355] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1930.454515] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1938.009509] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1942.905434] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1943.169872] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1947.856159] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1948.174363] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1951.221688] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1951.230624] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1951.244778] Lustre: Skipped 1 previous similar message [ 1951.256737] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1956.223522] Lustre: server umount lustre-OST0000 complete [ 1961.446195] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1961.461931] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1961.466824] Lustre: Skipped 1 previous similar message [ 1961.489494] Lustre: Skipped 2 previous similar messages [ 1966.568511] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1966.588825] Lustre: Skipped 1 previous similar message [ 1966.769171] Lustre: server umount lustre-MDT0000 complete [ 1970.408315] LustreError: 68372:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789299334 with bad export cookie 15195799122154753376 [ 1970.411277] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1970.416087] LustreError: 68372:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1970.760666] Lustre: server umount lustre-MDT0001 complete [ 1980.261720] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 07:35:42 (1789299342) [ 1992.723220] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 2039.137579] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_hostid [ 2047.819345] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 2087.603275] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2087.853687] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2087.873675] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2087.942128] Lustre: lustre-MDT0000: new disk, initializing [ 2088.033258] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2088.045623] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2092.409930] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 2103.545430] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2103.728449] Lustre: 76371:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2103.813442] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2103.818516] Lustre: Skipped 1 previous similar message [ 2103.926910] Lustre: lustre-MDT0001: new disk, initializing [ 2104.156488] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2104.257297] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2104.283162] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2110.287422] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 2119.999552] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2126.784429] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2134.176266] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2134.603358] Lustre: lustre-OST0000: new disk, initializing [ 2134.611907] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2134.623127] Lustre: 78340:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2134.733320] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2140.725567] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2140.728757] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2140.881649] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2141.780731] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 2151.833399] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2157.589622] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2157.903056] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2163.995368] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2164.216236] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2171.362754] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2171.370268] LustreError: Skipped 1 previous similar message [ 2171.377221] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2171.393598] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2173.229086] Lustre: server umount lustre-OST0000 complete [ 2181.102862] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2181.110889] LustreError: Skipped 1 previous similar message [ 2181.119679] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2181.146559] Lustre: Skipped 1 previous similar message [ 2181.180526] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2181.190404] Lustre: Skipped 1 previous similar message [ 2184.119872] Lustre: server umount lustre-MDT0000 complete [ 2188.604301] LustreError: 76363:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789299552 with bad export cookie 15195799122154754258 [ 2188.631066] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2189.182762] Lustre: server umount lustre-MDT0001 complete [ 2198.624257] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 07:39:21 (1789299561) [ 2206.177409] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2206.855646] LustreError: 80753:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2206.871320] LustreError: 80753:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2207.009520] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2212.327267] LustreError: 80754:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2212.553276] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 2217.445914] LustreError: 80753:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2222.190044] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2227.741336] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 2235.234778] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2241.563240] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2248.552801] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2248.887673] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2248.890699] Lustre: Skipped 1 previous similar message [ 2255.780572] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 2264.246373] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2268.399466] Lustre: 83426:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting bdae8c5c-67a3-4624-881c-b384230bddde at adminstrative request [ 2272.747394] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2272.769367] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2281.958603] Lustre: MGS: Client bdae8c5c-67a3-4624-881c-b384230bddde (at 0@lo) reconnecting [ 2281.966273] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xd2e252ca723a1444 to 0xd2e252ca723a16e4 [ 2281.975894] Lustre: MGC192.168.201.155@tcp: Connection restored to 0@lo (at 0@lo) [ 2306.850912] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2307.029430] Lustre: lustre-OST0001: new disk, initializing [ 2307.032899] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2307.041477] Lustre: 83817:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2314.023417] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 2314.889101] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2314.900307] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2315.000307] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2324.289041] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2330.600529] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2330.620790] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2330.633116] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2330.639907] Lustre: Skipped 2 previous similar messages [ 2330.655611] Lustre: Skipped 3 previous similar messages [ 2341.343134] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2341.457451] Lustre: server umount lustre-OST0001 complete [ 2345.954427] LustreError: 82578:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2345.956426] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2345.987576] LustreError: 82578:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2346.002195] Lustre: Skipped 1 previous similar message [ 2351.076430] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2351.084092] Lustre: Skipped 7 previous similar messages [ 2351.085849] LustreError: 82577:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2351.113044] LustreError: 82577:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2351.550386] Lustre: server umount lustre-OST0000 complete [ 2355.681105] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2355.686914] LustreError: Skipped 1 previous similar message [ 2361.833505] Lustre: server umount lustre-MDT0000 complete [ 2366.117336] LustreError: 80735:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789299730 with bad export cookie 15195799122154755812 [ 2366.130933] LustreError: 80735:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2366.137252] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2366.616893] Lustre: server umount lustre-MDT0001 complete [ 2375.821506] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 07:42:18 (1789299738) [ 2377.530599] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2379.195532] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 07:42:22 (1789299742) [ 2380.802913] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2382.849448] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 07:42:25 (1789299745) [ 2384.564428] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2386.357441] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 07:42:29 (1789299749) [ 2388.148364] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2390.463841] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 07:42:32 (1789299752) [ 2392.605748] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2399.188683] loop: module loaded [ 2399.215704] loop0: detected capacity change from 0 to 4294967296 [ 2399.331793] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2401.350632] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2405.229820] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2410.212842] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2441.671799] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2441.740355] LustreError: 87944:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2441.749515] LustreError: 87944:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2441.770594] LustreError: 87944:0:(tgt_mount.c:2582:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2441.782979] LustreError: 87944:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2463.470658] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_hostid [ 2475.421096] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 2515.570768] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2515.861981] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2515.886934] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2515.946245] Lustre: lustre-MDT0000: new disk, initializing [ 2516.011106] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2516.021866] Lustre: Skipped 1 previous similar message [ 2516.031779] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2521.298965] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 2533.097045] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2533.297635] Lustre: 90813:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2533.699788] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2538.382835] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 2548.770290] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2554.358904] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2561.351598] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2561.640341] Lustre: 92783:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2568.632137] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 2570.854657] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2570.961430] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2577.303456] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2584.224932] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2584.501652] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2591.054789] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2591.265771] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2596.839709] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2596.854705] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2596.863935] Lustre: Skipped 2 previous similar messages [ 2596.869886] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2596.878844] Lustre: Skipped 6 previous similar messages [ 2599.680809] Lustre: server umount lustre-OST0000 complete [ 2610.542800] Lustre: server umount lustre-MDT0000 complete [ 2612.199115] LustreError: 90824:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2612.221810] LustreError: 90824:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2614.975123] LustreError: 90804:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789299979 with bad export cookie 15195799122154756428 [ 2614.996542] LustreError: 90804:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2614.998115] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2615.399865] Lustre: server umount lustre-MDT0001 complete [ 2624.875493] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 07:46:27 (1789299987) [ 2632.578675] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2633.311943] LustreError: 95194:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2638.236124] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 2647.742913] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2648.115318] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2648.127867] Lustre: Skipped 3 previous similar messages [ 2653.640373] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 2662.519292] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2668.909587] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2676.003467] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2683.520574] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 2692.642752] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2698.173639] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2698.456859] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2705.019225] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2705.414065] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2714.226385] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2714.397741] Lustre: lustre-OST0001: new disk, initializing [ 2714.401060] Lustre: Skipped 2 previous similar messages [ 2714.404877] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2714.408580] Lustre: Skipped 3 previous similar messages [ 2714.413753] Lustre: 98566:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2721.668942] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 2721.958861] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2721.974481] Lustre: Skipped 2 previous similar messages [ 2721.999047] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2722.238901] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2732.089891] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2737.317696] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2737.571686] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2742.592390] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2742.965481] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2763.458921] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2785.400665] Lustre: Failing over lustre-MDT0000 [ 2785.865547] Lustre: server umount lustre-MDT0000 complete [ 2786.277722] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2786.298258] LustreError: Skipped 2 previous similar messages [ 2786.300169] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2786.326575] Lustre: Skipped 3 previous similar messages [ 2786.335314] LustreError: 96136:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2786.365671] LustreError: 96136:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2805.216788] Lustre: 66604:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789300152/real 1789300152] req@00000000a44d0df3 x1876216196120960/t0(0) o400->MGC192.168.201.155@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1789300168 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2805.255182] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2811.124276] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2811.126400] LDISKFS-fs (dm-0): recovery complete [ 2811.130759] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2815.457712] LustreError: 66600:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@0000000028526230 x1876216196129280/t0(0) o250->MGC192.168.201.155@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2816.055695] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2818.081216] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2818.082180] Lustre: 97399:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2821.097913] Lustre: 95195:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2821.103181] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2821.111451] Lustre: 95195:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2821.223069] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2821.503944] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 2821.569494] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2821.940742] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2821.949262] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2822.276072] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2822.285217] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2822.296224] Lustre: 101421:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2822.305944] Lustre: 101421:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2822.964863] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2823.004659] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2823.320453] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2823.331748] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2824.320092] Lustre: 101421:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2824.342829] Lustre: 101421:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 5 previous similar messages [ 2824.984230] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2825.001977] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2825.327128] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2825.337601] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2828.521629] Lustre: 101421:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2828.559546] Lustre: 101421:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2829.212847] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2829.222191] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2829.543179] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2829.551755] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2836.576983] Lustre: 101421:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2836.582330] Lustre: 101421:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 2837.224775] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2837.247376] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2837.576390] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2837.596297] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2852.867547] Lustre: 101421:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2852.878465] Lustre: 101421:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2853.521657] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2853.547756] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2853.871134] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2853.882590] LustreError: 101421:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2878.287209] Lustre: 101421:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2878.299350] Lustre: 101421:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2878.310967] Lustre: 101421:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client b4998a92-ff17-4310-b2f7-b49be65dc601@192.168.201.55@tcp [ 2878.328872] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2878.336806] Lustre: 101421:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@0000000056118149 x1876216171795712/t0(8589934775) o101->b4998a92-ff17-4310-b2f7-b49be65dc601@192.168.201.55@tcp:83/0 lens 592/0 e 0 to 0 dl 1789300253 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2878.375228] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2878.387830] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2878.400412] Lustre: lustre-MDT0000: Denying connection for new client b4998a92-ff17-4310-b2f7-b49be65dc601 (at 192.168.201.55@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2878.419342] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2878.434786] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2878.443750] Lustre: Skipped 2 previous similar messages [ 2878.468604] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 2878.475191] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 2892.787604] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2892.799091] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2892.809591] Lustre: Skipped 3 previous similar messages [ 2892.814567] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2892.833508] Lustre: Skipped 4 previous similar messages [ 2897.391643] Lustre: server umount lustre-OST0000 complete [ 2897.893125] LustreError: 97017:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2897.905591] LustreError: 97017:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 43 previous similar messages [ 2907.646450] Lustre: server umount lustre-OST0001 complete [ 2917.992516] Lustre: server umount lustre-MDT0000 complete [ 2922.323500] LustreError: 95174:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789300286 with bad export cookie 15195799122154807647 [ 2922.328252] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2922.341733] LustreError: 95174:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2922.715301] Lustre: server umount lustre-MDT0001 complete [ 2932.103606] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 07:51:34 (1789300294) [ 2940.860864] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2940.867319] Lustre: *** cfs_fail_loc=197, val=0*** [ 2956.511660] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2961.631567] LustreError: 103395:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.155@tcp: failed processing log, type 4: rc = -110 [ 2966.773449] Lustre: *** cfs_fail_loc=197, val=0*** [ 2966.781779] Lustre: Skipped 65 previous similar messages [ 2987.362202] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2987.365963] Lustre: Skipped 3 previous similar messages [ 2993.060887] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 3001.255467] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3003.879190] Lustre: server umount lustre-OST0000 complete [ 3013.051062] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 07:52:55 (1789300375) [ 3037.572559] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3038.017395] LustreError: 105015:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3038.036698] LustreError: 105015:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 3043.949389] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 3056.102664] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3061.127565] LustreError: 105046:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3061.138062] LustreError: 105046:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3061.149847] LustreError: 105046:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 23, retries 0, failed: rc = -5 [ 3061.565969] Lustre: server umount lustre-MDT0000 complete [ 3101.240406] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_hostid [ 3113.422174] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 3154.531755] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3154.812587] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3154.838707] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3154.932362] Lustre: lustre-MDT0000: new disk, initializing [ 3155.066585] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3160.192542] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 3170.965499] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3171.129423] Lustre: 109223:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3171.461368] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3177.669306] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 3189.827845] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3196.556672] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3202.868635] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3203.406242] Lustre: 111195:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3209.961032] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3211.044108] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 3223.371152] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3229.388795] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3229.636496] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3234.265543] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3234.520894] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3240.932272] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3240.933760] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3240.948453] LustreError: Skipped 1 previous similar message [ 3240.949762] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3240.967711] Lustre: Skipped 6 previous similar messages [ 3240.989197] Lustre: Skipped 6 previous similar messages [ 3242.617836] Lustre: server umount lustre-OST0000 complete [ 3253.037361] Lustre: server umount lustre-MDT0000 complete [ 3256.837876] LustreError: 109217:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789300620 with bad export cookie 15195799122154818280 [ 3256.839741] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3256.845555] LustreError: 109217:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3257.211412] Lustre: server umount lustre-MDT0001 complete [ 3266.214840] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 07:57:08 (1789300628) [ 3277.987193] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [ 3281.475570] Key type lgssc unregistered [ 3282.005243] LNet: 113683:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3282.026253] LNetError: 113683:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3282.070808] LNet: Removed LNI 192.168.201.155@tcp [ 3283.243088] Key type .llcrypt unregistered [ 3283.245077] Key type ._llcrypt unregistered [ 3303.259598] Key type ._llcrypt registered [ 3303.261658] Key type .llcrypt registered [ 3303.392927] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_hostid [ 3321.903293] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 3322.884526] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3322.923951] alg: No test for adler32 (adler32-zlib) [ 3324.391264] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [ 3324.655716] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [ 3326.327227] Key type lgssc registered [ 3327.709684] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3375.472857] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3375.496689] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3376.797457] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3376.847364] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3376.940764] Lustre: lustre-MDT0000: new disk, initializing [ 3377.004094] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3377.025502] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3382.308988] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 3394.510646] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3394.836509] Lustre: lustre-OST0000: new disk, initializing [ 3394.844280] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3394.852692] Lustre: Skipped 1 previous similar message [ 3394.860311] Lustre: 117853:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3394.987832] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3402.426603] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 3403.944559] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3403.957709] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3404.015195] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3413.668435] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3413.873905] Lustre: lustre-OST0001: new disk, initializing [ 3413.877749] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3413.885416] Lustre: 118715:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3413.996716] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3421.380557] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 3422.388442] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3422.401634] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3422.451252] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3432.933937] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3432.939489] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3432.950927] Lustre: Skipped 1 previous similar message [ 3435.543779] Lustre: server umount lustre-MDT0000 complete [ 3442.199538] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3447.113809] LustreError: 117852:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789300811 with bad export cookie 2266220326951182518 [ 3447.131163] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3447.136182] LustreError: 117852:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3447.202603] Lustre: server umount lustre-OST0000 complete [ 3451.287988] Lustre: server umount lustre-OST0001 complete [ 3461.043512] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 08:00:23 (1789300823) [ 3483.367313] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 08:00:46 (1789300846) [ 3492.963263] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 3517.615317] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 08:01:20 (1789300880) [ 3562.910868] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_hostid [ 3574.145596] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 3610.452768] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3610.702345] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3610.750095] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3610.896961] Lustre: lustre-MDT0000: new disk, initializing [ 3611.069131] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3611.108337] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3616.384407] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 3627.764286] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3627.843989] Lustre: 126484:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3627.908062] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3627.913781] Lustre: Skipped 1 previous similar message [ 3628.004660] Lustre: lustre-MDT0001: new disk, initializing [ 3628.126467] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3628.165715] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3628.175638] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3632.609984] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 3641.658666] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3647.012488] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3655.131187] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3655.487699] Lustre: lustre-OST0000: new disk, initializing [ 3655.492912] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3655.500632] Lustre: 128451:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3655.656487] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3662.416493] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3662.426095] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3662.578712] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3662.767714] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 3674.276377] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3679.321711] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3679.597875] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3685.085080] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3685.405989] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3688.426657] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3688.431545] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3688.456644] LustreError: Skipped 1 previous similar message [ 3688.472711] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3688.481092] Lustre: Skipped 1 previous similar message [ 3693.537869] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3693.553585] Lustre: Skipped 1 previous similar message [ 3693.850507] Lustre: server umount lustre-OST0000 complete [ 3698.656383] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3698.677082] Lustre: Skipped 1 previous similar message [ 3698.684937] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3698.697918] Lustre: Skipped 2 previous similar messages [ 3703.777091] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3703.787964] Lustre: Skipped 1 previous similar message [ 3704.375391] Lustre: server umount lustre-MDT0000 complete [ 3708.297451] LustreError: 126474:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789301072 with bad export cookie 2266220326951183736 [ 3708.298843] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3708.314096] LustreError: 126474:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3708.566015] Lustre: server umount lustre-MDT0001 complete [ 3719.853194] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 08:04:42 (1789301082) [ 3728.722122] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3729.498448] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3733.743430] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 3734.497042] LustreError: 130861:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3734.539548] LustreError: 130861:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3739.626120] LustreError: 130860:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3742.827596] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3748.461545] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 3757.827134] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3764.966284] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3773.388276] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3773.592720] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3773.596149] Lustre: Skipped 1 previous similar message [ 3779.456898] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 3786.275468] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3793.594568] Lustre: *** cfs_fail_loc=159, val=0*** [ 3809.935251] Lustre: lustre-MDT0000: Client 783f9818-a959-4bf1-b3ef-74e35c8ca89c (at 192.168.201.55@tcp) reconnecting [ 3811.738567] Lustre: *** cfs_fail_loc=159, val=0*** [ 3811.742039] Lustre: Skipped 5 previous similar messages [ 3828.294184] Lustre: lustre-MDT0000: Client 783f9818-a959-4bf1-b3ef-74e35c8ca89c (at 192.168.201.55@tcp) reconnecting [ 3834.336280] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3834.342197] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3834.353297] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3834.374347] Lustre: Skipped 2 previous similar messages [ 3834.384055] Lustre: Skipped 1 previous similar message [ 3834.913927] Lustre: server umount lustre-OST0000 complete [ 3839.471937] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3845.211145] Lustre: server umount lustre-MDT0000 complete [ 3849.257422] LustreError: 130842:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789301213 with bad export cookie 2266220326951184625 [ 3849.259570] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3849.269075] LustreError: 130842:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3849.755773] Lustre: server umount lustre-MDT0001 complete [ 3862.767921] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [ 3866.373867] Key type lgssc unregistered [ 3866.800083] LNet: 134640:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3866.805501] LNetError: 134640:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3866.831710] LNet: Removed LNI 192.168.201.155@tcp [ 3867.901994] Key type .llcrypt unregistered [ 3867.904584] Key type ._llcrypt unregistered [ 3896.534645] Key type ._llcrypt registered [ 3896.537343] Key type .llcrypt registered [ 3896.672581] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 3898.688086] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3898.782975] alg: No test for adler32 (adler32-zlib) [ 3900.164642] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [ 3900.639404] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [ 3902.383670] Key type lgssc registered [ 3904.237540] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3914.958902] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3914.976453] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3916.841134] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3923.352846] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 3926.008698] LustreError: 136054:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3926.037950] LustreError: 136054:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3931.108625] LustreError: 136055:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3934.753166] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3934.959962] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3939.792454] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 3948.571913] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3957.703962] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3966.895619] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3967.427228] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3974.572812] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 3982.679195] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3986.878476] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3999.037188] Lustre: *** cfs_fail_loc=159, val=0*** [ 4014.676707] Lustre: lustre-MDT0000: Client b36ed709-fa84-42fe-b53f-62b54949ac18 (at 192.168.201.55@tcp) reconnecting [ 4017.116243] Lustre: *** cfs_fail_loc=159, val=0*** [ 4033.672168] Lustre: lustre-MDT0001: Client b36ed709-fa84-42fe-b53f-62b54949ac18 (at 192.168.201.55@tcp) reconnecting [ 4035.500466] Lustre: *** cfs_fail_loc=159, val=0*** [ 4035.502145] Lustre: Skipped 3 previous similar messages [ 4051.013348] Lustre: lustre-MDT0001: Client b36ed709-fa84-42fe-b53f-62b54949ac18 (at 192.168.201.55@tcp) reconnecting [ 4055.410463] Lustre: 139113:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4057.973897] Lustre: *** cfs_fail_loc=159, val=0*** [ 4057.979818] Lustre: Skipped 4 previous similar messages [ 4074.606364] Lustre: lustre-MDT0000: Client 29b1d498-54f0-4488-acdc-67429fb0835d (at 192.168.201.55@tcp) reconnecting [ 4075.956093] Lustre: *** cfs_fail_loc=159, val=0*** [ 4075.957945] Lustre: Skipped 14 previous similar messages [ 4092.490029] Lustre: lustre-MDT0000: Client 29b1d498-54f0-4488-acdc-67429fb0835d (at 192.168.201.55@tcp) reconnecting [ 4094.300960] Lustre: 139403:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4113.895089] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4113.910614] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4113.920915] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4114.918453] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4114.936390] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4114.945721] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4118.964058] Lustre: server umount lustre-OST0000 complete [ 4124.640553] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4124.656359] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4124.677825] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4124.686027] Lustre: Skipped 1 previous similar message [ 4129.355884] Lustre: server umount lustre-MDT0000 complete [ 4130.282849] LustreError: 136055:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4130.303533] LustreError: 136055:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4133.142360] LustreError: 136034:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789301497 with bad export cookie 11579416590630952414 [ 4133.149212] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4133.161319] LustreError: 136034:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4133.456470] Lustre: server umount lustre-MDT0001 complete [ 4146.036481] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [ 4149.361922] Key type lgssc unregistered [ 4149.708288] LNet: 140479:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4149.725820] LNetError: 140479:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4150.763599] LNet: Removed LNI 192.168.201.155@tcp [ 4151.840713] Key type .llcrypt unregistered [ 4151.844341] Key type ._llcrypt unregistered [ 4179.945582] Key type ._llcrypt registered [ 4179.948627] Key type .llcrypt registered [ 4180.098759] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 4181.426687] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4181.504417] alg: No test for adler32 (adler32-zlib) [ 4182.727459] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [ 4182.980899] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [ 4184.687200] Key type lgssc registered [ 4185.747422] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4193.640173] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4193.661745] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4195.076833] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4199.261312] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 4205.543075] LustreError: 141893:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4205.569497] LustreError: 141893:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4210.463531] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4210.657327] LustreError: 141892:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4211.155062] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4215.670418] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 4222.396683] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4228.979184] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4237.021390] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4237.561058] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4240.632663] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4243.467170] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 4252.142256] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4256.000968] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4262.430996] Lustre: 144475:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4265.854237] Lustre: 144523:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4267.865584] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4269.151384] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4274.822862] Lustre: 144672:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4275.960613] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4277.172370] Lustre: 144770:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4281.847586] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4281.864032] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4281.882524] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4281.895915] LustreError: Skipped 1 previous similar message [ 4281.922535] Lustre: Skipped 1 previous similar message [ 4286.951179] LustreError: 143899:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4287.182309] Lustre: server umount lustre-OST0000 complete [ 4292.069726] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4292.081054] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4292.086825] Lustre: Skipped 2 previous similar messages [ 4292.101066] Lustre: Skipped 1 previous similar message [ 4297.186050] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4297.209270] Lustre: Skipped 1 previous similar message [ 4302.307576] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4302.315634] Lustre: Skipped 1 previous similar message [ 4305.375790] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4305.619275] Lustre: server umount lustre-MDT0000 complete [ 4307.428530] LustreError: 141892:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4307.460592] LustreError: 141892:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4309.545642] LustreError: 141872:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789301673 with bad export cookie 15520441103774559219 [ 4309.546539] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4309.580923] LustreError: 141872:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4309.926086] Lustre: server umount lustre-MDT0001 complete [ 4321.483743] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [ 4324.504437] Key type lgssc unregistered [ 4324.909605] LNet: 145831:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4324.914485] LNetError: 145831:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4324.924928] LNet: Removed LNI 192.168.201.155@tcp [ 4325.792310] Key type .llcrypt unregistered [ 4325.793688] Key type ._llcrypt unregistered [ 4355.474072] Key type ._llcrypt registered [ 4355.479665] Key type .llcrypt registered [ 4355.701608] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 4357.080307] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4357.274038] alg: No test for adler32 (adler32-zlib) [ 4358.731958] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [ 4358.923726] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [ 4360.575178] Key type lgssc registered [ 4362.005812] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4371.141690] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4371.168193] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4372.870855] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4378.321474] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 4382.199612] LustreError: 147242:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4382.238935] LustreError: 147242:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4387.308204] LustreError: 147243:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4387.383297] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4387.721433] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4390.658850] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 4399.663285] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4405.348150] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4412.982358] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4413.322186] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4417.461145] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4420.399386] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 4428.847743] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4432.823180] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4438.930888] Lustre: *** cfs_fail_loc=159, val=0*** [ 4438.937929] Lustre: Skipped 1 previous similar message [ 4455.488235] Lustre: lustre-MDT0000: Client b817ba1c-4520-4548-89fd-72cfeb5bf6d7 (at 192.168.201.55@tcp) reconnecting [ 4473.830485] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4473.844426] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4473.865936] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4473.880510] Lustre: Skipped 1 previous similar message [ 4478.971421] Lustre: server umount lustre-OST0000 complete [ 4484.066539] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4484.087344] Lustre: Skipped 1 previous similar message [ 4484.093963] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4489.185756] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4489.201316] Lustre: Skipped 2 previous similar messages [ 4489.906417] Lustre: server umount lustre-MDT0000 complete [ 4494.320545] LustreError: 147242:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4494.332336] LustreError: 147242:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4494.655114] LustreError: 147222:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789301858 with bad export cookie 14256773572183057864 [ 4494.669894] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4494.675743] LustreError: 147222:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4495.175995] Lustre: server umount lustre-MDT0001 complete [ 4510.537661] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [ 4516.534619] Key type lgssc unregistered [ 4517.067930] LNet: 150920:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4517.079908] LNetError: 150920:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4517.121728] LNet: Removed LNI 192.168.201.155@tcp [ 4519.052572] Key type .llcrypt unregistered [ 4519.062053] Key type ._llcrypt unregistered [ 4546.274100] Key type ._llcrypt registered [ 4546.277796] Key type .llcrypt registered [ 4546.472605] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 4547.718544] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4547.802233] alg: No test for adler32 (adler32-zlib) [ 4549.267213] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [ 4549.571890] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [ 4551.359185] Key type lgssc registered [ 4552.727582] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4561.371931] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4561.398664] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4563.055386] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4568.292938] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 4573.667531] LustreError: 152333:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4573.683538] LustreError: 152333:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4577.347505] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4577.792123] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4583.427625] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 4591.160209] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4596.594926] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4604.391744] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4604.821799] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4607.936799] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4611.425649] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 4618.548625] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4622.274387] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4642.257420] Lustre: lustre-MDT0001: evicting 978d1a9b-1358-408f-9706-3949e5cc9c4e (at 192.168.201.55@tcp) by administrative request [ 4642.264698] Lustre: Skipped 1 previous similar message [ 4673.507642] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4673.511163] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4673.520201] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4673.526962] Lustre: Skipped 1 previous similar message [ 4673.539324] Lustre: Skipped 1 previous similar message [ 4676.600632] Lustre: server umount lustre-OST0000 complete [ 4683.777911] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4683.800123] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4683.807071] Lustre: Skipped 1 previous similar message [ 4686.867954] Lustre: server umount lustre-MDT0000 complete [ 4688.864566] LustreError: 153066:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4688.876315] LustreError: 153066:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4690.713578] LustreError: 154183:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789302054 with bad export cookie 7511177303033811886 [ 4690.718866] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4690.729627] LustreError: 154183:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4691.122679] Lustre: server umount lustre-MDT0001 complete [ 4702.454907] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [ 4705.293486] Key type lgssc unregistered [ 4705.583173] LNet: 156403:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4705.598500] LNetError: 156403:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4706.662264] LNet: Removed LNI 192.168.201.155@tcp [ 4707.570605] Key type .llcrypt unregistered [ 4707.572234] Key type ._llcrypt unregistered [ 4785.367290] Key type ._llcrypt registered [ 4785.368898] Key type .llcrypt registered [ 4785.501269] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 4786.449239] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4786.549362] alg: No test for adler32 (adler32-zlib) [ 4787.658521] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [ 4787.923309] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [ 4789.615671] Key type lgssc registered [ 4790.958173] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4799.243883] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4799.274623] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4800.935434] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4806.628150] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 4811.249991] LustreError: 158893:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4811.269274] LustreError: 158893:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4816.072382] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4816.353633] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 4816.711251] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4823.032338] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 4831.190811] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4837.469126] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4844.263657] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4844.671809] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4846.721184] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4851.499224] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 4860.377902] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4861.060372] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4871.654424] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4871.655802] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4871.682892] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4875.224377] Lustre: server umount lustre-OST0000 complete [ 4881.890607] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4881.920509] Lustre: Skipped 1 previous similar message [ 4881.935443] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4881.948155] Lustre: Skipped 2 previous similar messages [ 4886.681165] Lustre: server umount lustre-MDT0000 complete [ 4887.012352] LustreError: 158888:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4887.025719] LustreError: 158888:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4891.118402] LustreError: 158887:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4891.130023] LustreError: 158887:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4891.762386] LustreError: 158873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789302255 with bad export cookie 1043592580595482788 [ 4891.771067] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4891.791073] LustreError: 158873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4892.313259] Lustre: server umount lustre-MDT0001 complete [ 4905.179711] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [ 4908.651940] Key type lgssc unregistered [ 4908.976666] LNet: 162432:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4908.982272] LNetError: 162432:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4909.017960] LNet: Removed LNI 192.168.201.155@tcp [ 4909.952789] Key type .llcrypt unregistered [ 4909.955413] Key type ._llcrypt unregistered [ 4935.850497] Key type ._llcrypt registered [ 4935.852942] Key type .llcrypt registered [ 4935.997703] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 4937.506088] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4937.691168] alg: No test for adler32 (adler32-zlib) [ 4938.794709] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [ 4939.254924] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [ 4941.055314] Key type lgssc registered [ 4942.167512] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4957.309483] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 08:25:20 (1789302320) [ 4973.494481] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_hostid [ 4986.965621] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 5024.275457] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5024.296990] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5025.665496] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5025.762921] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5025.876295] Lustre: lustre-MDT0000: new disk, initializing [ 5026.049791] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5026.095184] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5032.510291] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 5045.110044] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5045.320426] Lustre: 166691:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5045.431235] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5045.435856] Lustre: Skipped 1 previous similar message [ 5045.555572] Lustre: lustre-MDT0001: new disk, initializing [ 5045.698106] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5045.793500] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5045.816510] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5052.307446] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 5064.090759] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5071.130721] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5079.082815] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5079.553518] Lustre: lustre-OST0000: new disk, initializing [ 5079.557783] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5079.575797] Lustre: 168664:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5079.746908] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5087.345479] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5087.371695] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5087.616479] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5087.765504] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 5101.711573] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5118.435720] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5118.440081] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5118.447347] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5118.460075] Lustre: Skipped 1 previous similar message [ 5120.187478] Lustre: server umount lustre-OST0000 complete [ 5127.650750] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5127.662105] LustreError: Skipped 1 previous similar message [ 5127.675251] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5127.691187] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5127.703268] Lustre: Skipped 1 previous similar message [ 5128.677777] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5128.701342] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5128.709642] Lustre: Skipped 1 previous similar message [ 5130.828715] Lustre: server umount lustre-MDT0000 complete [ 5135.809523] LustreError: 168663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789302499 with bad export cookie 2133693666779311617 [ 5135.817324] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5135.826761] LustreError: 168663:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5136.369286] Lustre: server umount lustre-MDT0001 complete [ 5150.370983] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [ 5154.689778] Key type lgssc unregistered [ 5155.182494] LNet: 170579:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5155.190338] LNetError: 170579:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5155.211074] LNet: Removed LNI 192.168.201.155@tcp [ 5156.531542] Key type .llcrypt unregistered [ 5156.534175] Key type ._llcrypt unregistered [ 5184.567977] Key type ._llcrypt registered [ 5184.579935] Key type .llcrypt registered [ 5184.731915] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 5185.860193] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5186.012242] alg: No test for adler32 (adler32-zlib) [ 5187.419365] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [ 5187.893502] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [ 5189.640272] Key type lgssc registered [ 5191.000791] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5199.526436] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5199.550927] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5201.105271] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5206.413280] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 5211.618910] LustreError: 171991:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5211.629945] LustreError: 171991:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5214.917892] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5215.358910] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5220.721871] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 5229.239222] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5235.963551] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5243.327163] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5243.809434] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5245.920187] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5249.842678] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 5257.168936] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5266.757713] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5266.786083] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5266.808061] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5266.816957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.829844] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5267.805760] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5267.823701] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5267.841572] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5267.861264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.876830] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5268.912248] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5268.925198] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5268.936603] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5268.940942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.948122] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5269.896661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.893909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.677511] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5271.683623] Lustre: Skipped 2 previous similar messages [ 5271.704890] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5271.721716] Lustre: Skipped 2 previous similar messages [ 5271.733331] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5271.745889] Lustre: Skipped 2 previous similar messages [ 5271.758155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.773551] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5271.780824] Lustre: Skipped 2 previous similar messages [ 5272.896042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.693949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.655067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.666805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.571962] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5276.578085] Lustre: Skipped 4 previous similar messages [ 5276.593596] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5276.605902] Lustre: Skipped 4 previous similar messages [ 5276.610608] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5276.616327] Lustre: Skipped 4 previous similar messages [ 5276.620546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.632748] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5276.640516] Lustre: Skipped 4 previous similar messages [ 5277.678327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.564164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.648229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.685830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.782909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.917730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.984651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.007662] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5285.015418] Lustre: Skipped 7 previous similar messages [ 5285.046110] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5285.062510] Lustre: Skipped 7 previous similar messages [ 5285.066777] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5285.073953] Lustre: Skipped 7 previous similar messages [ 5285.078826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.089296] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5285.092895] Lustre: Skipped 7 previous similar messages [ 5285.971188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.988553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.965085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.953715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.935957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.095432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.898941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.914157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.942902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.730749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.550809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.408363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.506072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.754451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.716214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.542912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.465285] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5301.471714] Lustre: Skipped 16 previous similar messages [ 5301.486627] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5301.493714] Lustre: Skipped 16 previous similar messages [ 5301.497271] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5301.500164] Lustre: Skipped 16 previous similar messages [ 5301.504671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.523270] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5301.548037] Lustre: Skipped 16 previous similar messages [ 5302.433193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.457975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.218435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.188063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.198398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.232149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.368424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.435418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.337669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.201178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.197652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.607252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.677756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.799438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.985935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.050504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.058228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.023994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.116560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.919993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.125341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.115523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.050160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.916436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.852661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.621596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.584457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.297094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.200166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.932771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.854099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.757501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.756769] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5333.762168] Lustre: Skipped 32 previous similar messages [ 5333.779584] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5333.789502] Lustre: Skipped 32 previous similar messages [ 5333.793128] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5333.797330] Lustre: Skipped 32 previous similar messages [ 5333.800343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.812820] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5333.815133] Lustre: Skipped 32 previous similar messages [ 5334.592283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.705848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.664990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.583959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.412988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.465983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.268470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.363632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.463556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.567298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.719896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.704040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.627043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.654847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.560664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.728508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.748931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.808063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.722960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.047197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.948098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.180325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.337233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.683352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.615952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.802338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.693976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.859104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.941923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.991676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.971707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.172866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.259276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.269903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.249906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.372967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.677676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.787743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.932574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.794073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.033866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.979676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.288883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.131761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.283379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.281676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.392690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.360925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.521811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.351752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.401538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.243502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.294732] LustreError: 172011:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5389.305548] LustreError: 172011:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5389.320924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.195952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.054497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.067261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.215655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.125412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.194996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.174376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.249175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.254575] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5398.259137] Lustre: Skipped 61 previous similar messages [ 5398.293165] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5398.323183] Lustre: Skipped 61 previous similar messages [ 5398.331175] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5398.334394] Lustre: Skipped 61 previous similar messages [ 5398.347313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.358602] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5398.362270] Lustre: Skipped 61 previous similar messages [ 5399.483813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.468830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.568703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.596935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.601046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.495341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.371412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.312797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.297034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.182242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.063188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.078655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.009713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.220514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.184618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.228240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.044823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.852876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.548978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.392220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.363990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.275176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.105130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.083913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.062877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.040734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.813673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.222995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.321668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.489801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.558195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.574901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.948841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.984242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.878191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.727538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.775451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.623858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.505207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.448273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.375620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.270518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.306484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.239901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.291574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.531447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.555802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.684215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.688588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.522513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.411061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.568482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.951645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.311130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.358973] LustreError: 172011:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5453.365946] LustreError: 172011:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5453.403816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.443912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.515429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.521450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.565427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.417252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.319780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.514286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.392983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.250130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.217830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.359761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.390751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.321199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.311821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.315575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.135276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.835514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.773667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.834411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.020536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.061786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.123105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.852762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.799954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.062727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.848568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.062707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.331139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.797933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.868478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.695207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.885704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.906022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.735796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.895290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.862272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.363671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.648262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.605247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.560604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.287343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.151496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.253026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.144499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.273787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.277796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.134240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.990034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.871995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.737673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.722214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.481207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.466508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.294790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.165236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.197964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.008636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.969881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.113156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.035974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.001459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.135273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.980057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.948269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.693274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.386815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.021711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.691179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.512953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.380472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.163245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.086223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.129087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.082090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.204909] LustreError: 172011:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5526.223621] LustreError: 172011:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5526.236937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.303847] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5527.315634] Lustre: Skipped 130 previous similar messages [ 5527.345166] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5527.364889] Lustre: Skipped 130 previous similar messages [ 5527.372203] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5527.382497] Lustre: Skipped 130 previous similar messages [ 5527.391235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.404095] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5527.411847] Lustre: Skipped 130 previous similar messages [ 5528.208695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.253475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.100870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.013862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.180416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.986414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.775988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.863837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.009795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.929659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.052810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.043443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.073791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.961072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.970369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.811500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.868719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.952595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.930968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.093720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.078695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.063206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.166242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.055481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.665746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.260460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.993219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.789860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.797752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.713431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.689676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.619792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.514254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.367844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.137499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.284650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.260218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.235295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.271536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.102167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.078974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.025280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.834180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.854524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.904663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.811358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.734801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.738646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.615418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.556475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.448649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.415644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.461894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.773205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.999622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.239499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.364072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.259742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.464876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.734574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.649448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.596524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.532888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.646811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.616339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.629433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.372924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.451525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.494629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.580895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.464519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.443286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.363466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.493571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.313917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.289605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.373428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.243310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.247890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.159168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.248486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.246623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.144442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.837935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.562214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.332513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.219418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.212716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.965755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.881396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.827389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.880196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.817980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.794383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.735318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.642826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.762728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.701970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.864812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.988229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.861386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.889369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.747495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.772546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.639253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.697816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.589128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.530231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.513069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.434814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.166902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.077347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.111391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.903085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.811837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.668906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.696643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.529346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.418317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.288933] LustreError: 172011:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5642.297903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.299927] LustreError: 172011:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5642.991015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.847812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.698601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.523380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.248962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.073325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.736251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.690236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.530883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.474159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.383474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.367855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.250198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.122632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.128795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.065931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.019905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.012297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.074451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.082632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.041640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.022793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.952790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.743711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.628298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.581784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.573344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.453708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.416530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.494445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.331474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.347585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.337880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.295657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.333663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.385058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.244599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.216656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.304593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.060260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.025671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.770445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.697669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.584392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.386548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.266626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.207698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.139288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.029283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.000853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.898958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.025621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.998978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.939036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.664240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.287673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.951132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.633273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.642375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.564765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.543468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.427119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.325304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.400719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.104509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.137448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.228498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.127200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.794876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.602833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.474287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.425452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.330052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.199789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.916692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.878700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.653064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.474849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.477066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.074725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.579821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.007073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.349803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.357239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.140112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.831356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.851520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.847474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.501113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.458683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.573962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.611572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.481632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.196811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.932575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.058635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.158635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.087755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.062832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.224336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.283091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.294178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.294503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.353052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.119068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.984132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.997376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.026313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.928101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.933386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.643637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.534764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.528365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.386337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.149324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.023957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.013423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.985822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.041745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.033128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.127864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.956306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.005894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.892371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.978959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.684877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.875725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.733545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.739923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.616710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.465815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.240194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.055354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.733731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.552799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.254349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.121447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.910914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.764662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.857067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.873483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.834553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.804057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.760495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.613446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.543383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.402021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.252757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.142331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.080902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.049759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.898950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.806111] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5783.809736] Lustre: Skipped 272 previous similar messages [ 5783.821282] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5783.828992] Lustre: Skipped 272 previous similar messages [ 5783.833784] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5783.839361] Lustre: Skipped 272 previous similar messages [ 5783.842860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.851532] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5783.857528] Lustre: Skipped 272 previous similar messages [ 5784.662364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.617992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.642549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.656467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.665827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.583376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.550296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.356969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.188882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.987860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.913238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.893850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.723507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.541785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.492073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.397128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.563861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.471322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.496247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.457220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.327124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.167234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.152602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.936396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.880447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.799418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.743353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.408490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.152839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.000732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.928046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.719497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.577714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.368777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.305347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.303837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.260684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.392990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.265639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.302117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.993195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.061518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.874599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.712979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.533400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.693042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.630705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.555478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.180700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.116544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.806029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.767920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.612950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.387708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.255631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.291581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.414813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.484344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.800551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.216973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.138647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.170215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.172826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.001380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.027277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.853857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.755130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.528647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.454772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.477521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.517410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.426832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.307461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.216567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.270702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.217186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.004577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.621508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.484674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.116200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.803234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.455183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.157465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.849212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.669972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.471761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.253389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.131129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.023304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.839411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.850791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.570231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.498327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.387647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.321109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.182972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.057324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.069303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.172055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.953647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.909429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.770758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.583912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.382311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.210393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.119768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.832194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.762883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.662963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.575505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.420478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.366795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.269292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.084645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.864821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.829520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.736746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.517198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.495654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.318296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.265588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.090701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.043214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.754076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.855813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.981885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.913343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.820520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.976790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.657485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.450801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.197070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.051669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.836817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.750227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.682911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.592407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.453448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.268888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.971858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.807413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.987570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.916386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.661452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.521551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.423738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.235369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.185344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.904185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.969266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.876768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.880909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.813647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.919595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.790825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.927610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.941156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.800238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.743988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.686275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.955202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.050736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.260803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.372379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.240525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.386347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.467804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.470492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.319594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.609384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.917565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.292661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.563469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.712577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.727952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.156449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.411614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.187941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.395945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.420598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.594665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.886197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.742858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.711544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.494497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.247842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.905677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.793558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.498838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.576369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.546360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.315725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.058150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.212197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.021564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.092180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.173867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.113361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.104642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.997842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.765104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.893816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.893479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.969252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.127692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.988092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.830844] LustreError: 172011:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5975.838919] LustreError: 172011:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5975.844708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.742154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.631327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.637649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.776543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.799668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.787435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.055647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.140972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.971433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.996784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.119388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.248112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.235598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.314073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.377980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.112171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.766697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.782985] LustreError: 172011:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5993.800614] LustreError: 172011:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5993.817693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.727863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.835435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.874444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.771281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.663682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.563801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.257904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.292879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.178214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.308200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.318092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.476447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.643523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.939604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.174154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.212063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.157534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.143838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.564146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.931879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.171253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.322322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.483408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.737465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.812087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.708150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.901272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.861880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.917097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.724529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.042449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.199646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.218876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.154899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.095400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.218485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.088398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.209857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.779839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.711531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.909368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.117346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.957790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.014359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.010519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.046777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.011335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.783917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.590206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.761253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.613473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.596988] LustreError: 172011:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6047.617834] LustreError: 172011:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6047.619969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.398591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.303591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.009201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.886616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.704585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.760762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.767329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.727396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.718101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.766572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.628500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.344078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.106183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.933694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.748711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.597977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.543243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.488198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.506701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.432028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.275729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.318267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.271976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.308786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.298386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.324872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.243448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.248484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.219688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.151268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.031466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.881767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.856339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.778715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.889580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.818533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.748093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.528501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.440181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.358844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.332369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.117708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.004982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.652974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.513196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.381294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.326283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.135637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.995293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.844833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.664492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.522632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.425489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.192214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.933565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.756067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.609361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.466644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.213608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.063169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.929320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.640930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.364648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.304821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.166236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.084749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.793754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.582667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.457305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.120547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.886025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.930685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.740617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.505656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.441253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.392136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.389979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.470578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.455205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.387888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.274373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.265508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.979373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.865381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.699363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.624323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.388524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.250870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.075348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.786393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.618854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.484379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.485470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.537406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.464618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.457844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.488081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.462113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.607083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.393866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.282676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.285259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.262397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.138764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.140105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.380259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.188715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.977020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.013620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.865400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.926369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.713610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.404678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.133892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.120969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.018651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.030231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.200369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.154734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.168840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.220603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.544189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.598756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.550980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.533277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.209466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.856223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.002798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.069822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.182056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.265699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.097838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.108410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.078541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.176991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.229499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.211365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.235727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.430772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.382647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.352237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.366302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.470363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.370439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.465239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.439643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.535347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.650930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.741678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.715343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.903599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.955651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.024780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.110260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.315161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.351617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.628503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.780914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.702202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.688241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.830128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.806739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.726778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.602040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.391272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.322670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.156469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.093666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.967298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.821710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.557562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.369035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.105905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.060064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.919801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.925599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.114964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.987652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.912451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.708732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.880355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.854913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.886517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.856163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.827424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.358299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.194643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.173402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.433474] LustreError: 173813:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6542.448315] LustreError: 172011:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6960.097196] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6960.101872] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6960.114570] Lustre: Skipped 465 previous similar messages [ 6960.133727] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6961.140829] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6964.794888] Lustre: server umount lustre-OST0000 complete [ 6971.372021] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6971.381185] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6971.398397] Lustre: Skipped 2 previous similar messages [ 6974.576717] Lustre: server umount lustre-MDT0000 complete [ 6976.482440] LustreError: 171991:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6976.511893] LustreError: 171991:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6978.795206] LustreError: 222372:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789304342 with bad export cookie 9734175935752643464 [ 6978.799945] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6978.812028] LustreError: 222372:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6979.150465] Lustre: server umount lustre-MDT0001 complete [ 6992.658373] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [ 6996.140290] Key type lgssc unregistered [ 6996.479954] LNet: 227202:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6996.490286] LNetError: 227202:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6996.531183] LNet: Removed LNI 192.168.201.155@tcp [ 6997.759698] Key type .llcrypt unregistered [ 6997.761249] Key type ._llcrypt unregistered [ 7023.167277] Key type ._llcrypt registered [ 7023.170304] Key type .llcrypt registered [ 7023.279615] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 7024.655763] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7024.697830] alg: No test for adler32 (adler32-zlib) [ 7025.822278] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [ 7026.171046] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [ 7027.863170] Key type lgssc registered [ 7029.225950] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7037.426300] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7037.484351] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7039.188458] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7044.159115] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7049.696245] LustreError: 228612:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7049.718564] LustreError: 228612:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7053.074345] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7053.520222] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7058.260377] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7065.841233] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7071.251882] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7077.620668] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7077.944223] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7083.892926] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7084.099928] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7091.217372] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7097.174908] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7099.331321] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7099.344993] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7099.358459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7099.374937] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7099.376028] LustreError: 228632:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7104.485166] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7104.488441] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7104.507378] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7104.549544] Lustre: Skipped 1 previous similar message [ 7107.435263] Lustre: server umount lustre-OST0000 complete [ 7119.423231] LustreError: 228632:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7142.814626] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7143.089733] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7144.109951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7144.124477] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7144.140450] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7144.150551] Lustre: Skipped 1 previous similar message [ 7150.131613] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7158.923764] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7160.708232] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7167.011441] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff97065076a800.ost_server_uuid 50 [ 7168.619564] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff97065076a800.ost_server_uuid in IDLE state after 0 sec [ 7174.113387] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7174.117887] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7174.128764] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7174.131898] Lustre: Skipped 2 previous similar messages [ 7175.140180] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7175.141769] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7175.153787] Lustre: Skipped 1 previous similar message [ 7180.022927] Lustre: server umount lustre-OST0000 complete [ 7185.377042] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7185.382707] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7185.406222] Lustre: Skipped 2 previous similar messages [ 7185.421736] Lustre: Skipped 1 previous similar message [ 7190.484691] Lustre: server umount lustre-MDT0000 complete [ 7190.516907] LustreError: 228613:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7195.078717] LustreError: 228593:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789304559 with bad export cookie 1441907845907801341 [ 7195.089608] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7195.092033] LustreError: 228593:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7195.585196] Lustre: server umount lustre-MDT0001 complete [ 7208.166937] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [ 7211.350437] Key type lgssc unregistered [ 7211.724798] LNet: 233836:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7211.736599] LNetError: 233836:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7211.778663] LNet: Removed LNI 192.168.201.155@tcp [ 7213.317979] Key type .llcrypt unregistered [ 7213.327582] Key type ._llcrypt unregistered [ 7247.380676] Key type ._llcrypt registered [ 7247.383682] Key type .llcrypt registered [ 7247.534776] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 7248.864135] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7249.183879] alg: No test for adler32 (adler32-zlib) [ 7250.321381] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [ 7250.737616] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [ 7252.519455] Key type lgssc registered [ 7253.914613] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7263.386240] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7263.401970] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7264.928602] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7269.634892] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7275.493914] LustreError: 235561:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7275.532938] LustreError: 235561:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7278.249098] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7279.072740] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7284.064739] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7287.276456] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7293.496803] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7293.812482] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7300.902715] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7309.973478] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7310.126609] Lustre: lustre-OST0001: new disk, initializing [ 7310.136136] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7310.142386] Lustre: 237868:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7310.248455] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7311.315499] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7315.975834] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7320.204539] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7320.213078] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7320.278140] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7327.160626] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7339.588123] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7353.038504] Lustre: server umount lustre-MDT0000 complete [ 7355.884912] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7355.890456] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7355.901222] LustreError: 235561:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7356.218164] LustreError: 237867:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789304720 with bad export cookie 6284807514890273306 [ 7356.225314] LustreError: 237867:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7356.225630] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7356.397854] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7356.401830] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7356.419138] Lustre: Skipped 3 previous similar messages [ 7358.696521] Lustre: server umount lustre-MDT0001 complete [ 7362.603512] Lustre: server umount lustre-OST0000 complete [ 7366.613983] Lustre: server umount lustre-OST0001 complete [ 7387.604381] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 7397.059687] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7397.480205] LustreError: 240976:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7397.490474] LustreError: 240976:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7397.566406] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7402.550859] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7402.978096] LustreError: 240977:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7407.076103] LustreError: 240976:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7411.102729] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7411.544544] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7416.052941] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7419.284713] Lustre: 242117:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7425.442246] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7431.567903] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7436.008400] LustreError: 242472:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7439.321254] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7439.524165] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7439.532019] Lustre: Skipped 1 previous similar message [ 7442.603960] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 7442.610768] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7445.998604] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7453.654587] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7457.203897] Lustre: 243990:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7468.006423] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7468.012177] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7468.029789] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7468.041600] Lustre: Skipped 1 previous similar message [ 7472.719048] Lustre: server umount mylustre-MDT0000 complete [ 7473.635445] LustreError: 243296:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7473.661588] LustreError: 243296:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7476.370620] LustreError: 240951:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789304840 with bad export cookie 6284807514890276099 [ 7476.384051] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7476.386125] LustreError: 240951:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7476.748397] Lustre: server umount mylustre-MDT0001 complete [ 7491.393590] Lustre: server umount mylustre-OST0000 complete [ 7494.623281] Lustre: 234822:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789304842/real 1789304842] req@ffff947bda67f480 x1876221875078912/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789304858 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7494.641539] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7494.648040] Lustre: Skipped 3 previous similar messages [ 7495.297362] Lustre: server umount mylustre-OST0001 complete [ 7519.315054] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 7529.806023] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7530.140126] LustreError: 246300:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7530.268233] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7535.192597] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7542.885113] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7548.220529] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7551.057527] Lustre: 247441:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7557.349359] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7564.599199] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7572.935703] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7575.221392] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 7575.238596] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49792) [ 7578.947318] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7585.744270] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7588.899508] Lustre: 249314:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7594.476958] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7594.481630] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7594.500270] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7594.509096] Lustre: Skipped 3 previous similar messages [ 7600.867918] Lustre: server umount tfs-MDT0000 complete [ 7601.124125] LustreError: 247655:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7601.138299] LustreError: 247655:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 7604.122378] LustreError: 246277:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789304968 with bad export cookie 6284807514890277989 [ 7604.125985] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7604.130942] LustreError: 246277:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7604.660653] Lustre: server umount tfs-MDT0001 complete [ 7618.712291] Lustre: server umount tfs-OST0000 complete [ 7632.763039] Lustre: server umount tfs-OST0001 complete [ 7653.078853] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 7662.613215] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7662.985231] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7662.988758] Lustre: Skipped 3 previous similar messages [ 7667.046910] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7675.889302] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7681.369396] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7684.364485] Lustre: 252705:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7690.401099] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7696.176244] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7704.182330] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7708.539713] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 7708.557853] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49824) [ 7709.835340] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7716.963611] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7720.653192] Lustre: 254576:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7726.436903] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 09:11:29 (1789305089) [ 7742.951348] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7742.964075] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7742.976921] Lustre: Skipped 3 previous similar messages [ 7742.989569] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7742.992326] Lustre: Skipped 4 previous similar messages [ 7749.606520] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7749.614842] Lustre: Skipped 7 previous similar messages [ 7756.255579] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7758.513043] Lustre: server umount lustre-MDT0000 complete [ 7759.848032] LustreError: 252562:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7759.875555] LustreError: 252562:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 7762.274411] LustreError: 251540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789305126 with bad export cookie 6284807514890279655 [ 7762.274986] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7762.286145] LustreError: 251540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7764.963552] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7764.990919] Lustre: Skipped 3 previous similar messages [ 7765.003370] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7765.017399] Lustre: Skipped 5 previous similar messages [ 7768.667601] Lustre: server umount lustre-MDT0001 complete [ 7778.599934] Lustre: server umount lustre-OST0000 complete [ 7788.470761] Lustre: server umount lustre-OST0001 complete [ 7796.106194] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_hostid [ 7804.802770] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 7836.283541] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7836.497550] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7836.521404] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7836.583380] Lustre: lustre-MDT0000: new disk, initializing [ 7836.654960] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7836.657809] Lustre: Skipped 3 previous similar messages [ 7836.675476] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7840.878302] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7849.649709] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7849.817342] Lustre: 257763:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7849.886204] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7849.889818] Lustre: Skipped 1 previous similar message [ 7849.988237] Lustre: lustre-MDT0001: new disk, initializing [ 7850.185175] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7850.195678] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7854.669278] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7863.590695] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7863.832735] Lustre: lustre-OST0000: new disk, initializing [ 7863.841942] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7863.850941] Lustre: 259342:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7864.954904] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7864.965299] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7865.143449] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7868.873954] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7878.222528] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7878.355230] Lustre: lustre-OST0001: new disk, initializing [ 7878.357604] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7878.361943] Lustre: 260211:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7879.843867] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7879.854959] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7879.973661] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7884.241507] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 7894.415377] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 09:14:17 (1789305257) [ 7905.156533] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 09:14:28 (1789305268) [ 7921.127269] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7921.131108] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7921.138723] Lustre: Skipped 2 previous similar messages [ 7921.151093] Lustre: Skipped 3 previous similar messages [ 7925.035655] Lustre: server umount lustre-OST0000 complete [ 7933.805972] Lustre: server umount lustre-MDT0000 complete [ 7936.707770] LustreError: 257755:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789305300 with bad export cookie 6284807514890281209 [ 7936.717245] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7936.722496] LustreError: 257755:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7951.327489] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7951.598224] Lustre: server umount lustre-MDT0001 complete [ 7960.896470] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [ 8038.543309] Lustre: server umount lustre-OST0001 complete [ 8043.401700] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_hostid [ 8054.463395] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 8084.703960] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8084.904453] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8084.934102] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8085.029959] Lustre: lustre-MDT0000: new disk, initializing [ 8085.129596] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8089.587890] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 8098.330652] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8098.461221] Lustre: 265044:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8098.744437] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8098.749944] Lustre: Skipped 4 previous similar messages [ 8098.788538] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8101.956881] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 8109.831159] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8114.583701] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8120.179746] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8120.367258] Lustre: lustre-OST0000: new disk, initializing [ 8120.369531] Lustre: Skipped 1 previous similar message [ 8120.372649] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8120.374952] Lustre: Skipped 2 previous similar messages [ 8120.378531] Lustre: 267014:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8122.060350] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8122.067118] Lustre: Skipped 1 previous similar message [ 8122.072116] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8122.171410] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8125.440589] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 8132.924627] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8142.820556] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8142.821820] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8142.822984] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8142.822992] Lustre: Skipped 5 previous similar messages [ 8142.830195] LustreError: Skipped 1 previous similar message [ 8142.853668] Lustre: Skipped 5 previous similar messages [ 8143.965548] Lustre: server umount lustre-OST0000 complete [ 8153.361717] Lustre: server umount lustre-MDT0000 complete [ 8156.716942] LustreError: 265039:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789305520 with bad export cookie 6284807514890282679 [ 8156.726486] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8156.977474] Lustre: server umount lustre-MDT0001 complete [ 8167.089410] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [ 8169.681215] Key type lgssc unregistered [ 8169.957648] LNet: 268772:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8169.965975] LNetError: 268772:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8169.993933] LNet: Removed LNI 192.168.201.155@tcp [ 8170.649628] Key type .llcrypt unregistered [ 8170.652085] Key type ._llcrypt unregistered [ 8196.205966] Key type ._llcrypt registered [ 8196.208579] Key type .llcrypt registered [ 8196.373943] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 8197.325782] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8197.582059] alg: No test for adler32 (adler32-zlib) [ 8198.695387] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [ 8198.917469] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [ 8200.583429] Key type lgssc registered [ 8201.576508] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8208.244452] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8208.289099] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8209.614620] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8213.823355] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 8219.106099] LustreError: 270237:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8219.131561] LustreError: 270237:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8220.867522] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8221.178514] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8225.405354] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 8231.361430] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8235.779670] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8240.974372] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8241.226846] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8246.201362] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 8252.102909] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8252.941288] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8254.594083] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8256.995876] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8257.008070] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8257.025664] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8260.085203] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8260.090689] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8260.102986] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8260.128970] Lustre: Skipped 1 previous similar message [ 8261.042744] Lustre: server umount lustre-OST0000 complete [ 8265.187706] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8265.200067] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8265.200231] Lustre: Skipped 1 previous similar message [ 8265.212200] Lustre: Skipped 1 previous similar message [ 8270.308419] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8270.686984] Lustre: server umount lustre-MDT0000 complete [ 8274.230210] LustreError: 270217:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789305638 with bad export cookie 10540644080005213758 [ 8274.246567] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8274.255759] LustreError: 270217:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8274.565432] Lustre: server umount lustre-MDT0001 complete [ 8285.061837] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [ 8287.422781] Key type lgssc unregistered [ 8287.678891] LNet: 273818:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8287.687398] LNetError: 273818:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8287.734430] LNet: Removed LNI 192.168.201.155@tcp [ 8288.525717] Key type .llcrypt unregistered [ 8288.527536] Key type ._llcrypt unregistered [ 8302.633219] Key type ._llcrypt registered [ 8302.634874] Key type .llcrypt registered [ 8302.739800] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 8303.869164] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8303.898121] alg: No test for adler32 (adler32-zlib) [ 8304.889663] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [ 8305.072432] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [ 8306.719208] Key type lgssc registered [ 8307.954313] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8315.813581] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8315.825528] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8331.359560] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8336.479251] LustreError: 274988:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.155@tcp: failed processing log, type 4: rc = -110 [ 8363.167212] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8368.729879] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 8375.301592] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8377.419815] Lustre: server umount lustre-OST0000 complete [ 8392.828565] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [ 8394.897725] Key type lgssc unregistered [ 8395.143689] LNet: 276619:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8395.150476] LNetError: 276619:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8395.169900] LNet: Removed LNI 192.168.201.155@tcp [ 8395.806545] Key type .llcrypt unregistered [ 8395.808111] Key type ._llcrypt unregistered [ 8427.193743] Key type ._llcrypt registered [ 8427.201268] Key type .llcrypt registered [ 8427.304986] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 8427.968029] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8428.046827] alg: No test for adler32 (adler32-zlib) [ 8429.078316] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [ 8429.299048] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [ 8430.951179] Key type lgssc registered [ 8431.778821] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8442.484611] loop0: detected capacity change from 0 to 204800000 [ 8442.697305] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8442.705195] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8442.998079] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8444.011557] loop0: detected capacity change from 0 to 204800000 [ 8444.280291] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8444.297762] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8444.510751] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8445.347641] loop0: detected capacity change from 0 to 204800000 [ 8445.531732] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8445.555131] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8445.638137] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8446.590177] loop0: detected capacity change from 0 to 204800000 [ 8446.759441] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8447.688520] loop0: detected capacity change from 0 to 204800000 [ 8447.715979] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8451.412467] loop0: detected capacity change from 0 to 204800000 [ 8451.476026] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8454.343976] loop0: detected capacity change from 0 to 204800000 [ 8454.366917] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8457.365204] loop0: detected capacity change from 0 to 204800000 [ 8457.419454] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8460.539495] loop0: detected capacity change from 0 to 204800000 [ 8460.676889] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8460.739170] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8460.811281] Lustre: 280029:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8460.820278] Lustre: 280029:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8460.831384] Lustre: 280029:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8460.848493] Lustre: 280029:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8460.875655] Lustre: 280029:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8460.889035] Lustre: 280029:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8460.897423] Lustre: 280029:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8460.908500] Lustre: 280029:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8460.921595] Lustre: 280029:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8460.934395] Lustre: 280029:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8460.945460] Lustre: 280029:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8460.953619] Lustre: 280029:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8460.960648] Lustre: 280029:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8460.967383] Lustre: 280029:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8465.699355] Lustre: server umount MGS complete [ 8466.370622] loop0: detected capacity change from 0 to 204800000 [ 8466.470101] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8466.834551] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8466.877889] Lustre: lustre-MDT0000: Aborting client recovery [ 8467.571685] loop1: detected capacity change from 0 to 204800000 [ 8467.718059] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8467.764966] Lustre: 280459:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8467.772242] Lustre: 280459:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8467.778665] Lustre: 280459:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8467.784829] Lustre: 280459:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8467.791347] Lustre: 280459:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8467.803503] Lustre: 280459:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8467.812386] Lustre: 280459:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8467.845524] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8468.094862] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8468.154232] Lustre: lustre-MDT0001: Aborting client recovery [ 8468.813089] loop2: detected capacity change from 0 to 204800000 [ 8468.968394] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8469.072671] Lustre: 280537:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8469.090924] Lustre: 280537:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8469.096502] Lustre: 280537:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8469.107185] Lustre: 280537:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8469.115648] Lustre: 280537:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8469.124267] Lustre: 280537:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8469.131832] Lustre: 280537:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8469.284840] Lustre: Found index 0 for lustre-OST0000, updating log [ 8469.312246] Lustre: 280556:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8469.365936] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8469.376990] Lustre: lustre-OST0000: Aborting recovery [ 8470.090196] loop3: detected capacity change from 0 to 204800000 [ 8470.238266] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8470.342649] Lustre: 280621:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8470.349821] Lustre: 280621:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8470.356357] Lustre: 280621:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8470.363509] Lustre: 280621:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8470.370511] Lustre: 280621:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8470.379272] Lustre: 280621:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8470.390647] Lustre: 280621:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8470.413592] Lustre: Found index 1 for lustre-OST0001, updating log [ 8470.438757] Lustre: 280628:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8470.498985] Lustre: lustre-OST0001: Aborting recovery [ 8475.658121] LustreError: 280402:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -108 [ 8476.645611] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8476.659575] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8476.674603] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8476.693116] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8478.693936] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8478.695842] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8482.867522] LustreError: 280369:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 347us: rc = -2 [ 8483.371233] LustreError: 280368:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 485us: rc = -2 [ 8483.379371] LustreError: 280368:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 39 previous similar messages [ 8498.657566] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8498.668571] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8498.702786] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8499.170681] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8499.187242] Lustre: Skipped 1 previous similar message [ 8504.293160] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8504.305546] Lustre: Skipped 7 previous similar messages [ 8509.407959] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8509.408796] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8509.416610] Lustre: Skipped 3 previous similar messages [ 8509.622399] Lustre: server umount lustre-MDT0000 complete [ 8510.266411] LustreError: 280353:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789305874 with bad export cookie 16334324352124218658 [ 8510.275413] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8510.279538] LustreError: 280353:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8514.528667] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8514.529574] LustreError: 280862:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8514.535072] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8514.546509] Lustre: Skipped 2 previous similar messages [ 8514.563806] LustreError: 280862:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8515.552511] LustreError: 280367:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8515.566066] LustreError: 280367:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8516.832778] Lustre: server umount lustre-MDT0001 complete [ 8523.843200] Lustre: server umount lustre-OST0000 complete [ 8530.828398] Lustre: server umount lustre-OST0001 complete [ 8537.647856] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 09:25:00 (1789305900) [ 8561.568774] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_hostid [ 8569.755748] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 8596.466136] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8596.688585] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8596.709410] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8596.788154] Lustre: lustre-MDT0000: new disk, initializing [ 8596.847749] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8596.852242] Lustre: Skipped 1 previous similar message [ 8596.861988] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8600.627431] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 8608.507888] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8608.596432] Lustre: 284694:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8608.646986] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8608.650709] Lustre: Skipped 1 previous similar message [ 8608.704442] Lustre: lustre-MDT0001: new disk, initializing [ 8608.797728] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8608.836052] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8608.845572] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8611.969893] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 8618.408726] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8622.034886] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8626.084118] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8626.307627] Lustre: lustre-OST0000: new disk, initializing [ 8626.310934] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8626.316545] Lustre: 286661:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8626.389031] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8630.408879] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 8634.419890] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8634.425369] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8634.500851] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8637.212200] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8641.148952] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8645.457518] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8653.479993] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8661.814282] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8700.899025] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8700.910709] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8700.913159] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8700.917904] Lustre: Skipped 1 previous similar message [ 8700.925225] Lustre: Skipped 4 previous similar messages [ 8705.265390] Lustre: server umount lustre-OST0000 complete [ 8711.136746] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8711.144280] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8711.150064] Lustre: Skipped 1 previous similar message [ 8711.167331] Lustre: Skipped 1 previous similar message [ 8713.834938] Lustre: server umount lustre-MDT0000 complete [ 8716.268340] LustreError: 284705:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8716.279365] LustreError: 284705:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8716.291466] LustreError: 288170:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789306080 with bad export cookie 16334324352124221367 [ 8716.293824] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8716.296681] LustreError: 288170:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8716.515906] Lustre: server umount lustre-MDT0001 complete [ 8722.547389] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8725.630350] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 8728.576840] Lustre: server umount MGS complete [ 8735.771385] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8736.092790] LustreError: 290479:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8736.113561] LustreError: 290492:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8736.148173] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8739.176958] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 8741.353298] LustreError: 290480:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8745.185558] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8748.690848] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 8754.165705] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8758.457624] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8764.089833] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8769.094615] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 8775.590566] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8816.609820] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8816.623344] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8816.639641] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8820.463561] Lustre: server umount lustre-OST0000 complete [ 8827.364077] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8827.368995] LustreError: Skipped 1 previous similar message [ 8829.245780] Lustre: server umount lustre-MDT0000 complete [ 8831.745812] LustreError: 293334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789306195 with bad export cookie 16334324352124223292 [ 8831.750978] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8832.031971] Lustre: server umount lustre-MDT0001 complete [ 8838.852306] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [ 8840.847205] Key type lgssc unregistered [ 8841.063883] LNet: 294444:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8841.068988] LNetError: 294444:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8842.084918] LNet: Removed LNI 192.168.201.155@tcp [ 8842.665600] Key type .llcrypt unregistered [ 8842.667141] Key type ._llcrypt unregistered [ 8865.044852] Key type ._llcrypt registered [ 8865.046866] Key type .llcrypt registered [ 8865.126663] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_hostid [ 8875.280630] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 8875.905619] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8876.021944] alg: No test for adler32 (adler32-zlib) [ 8877.001456] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [ 8877.159776] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [ 8878.784982] Key type lgssc registered [ 8879.649371] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8904.771588] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8904.788656] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8905.950610] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8905.967679] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8906.031908] Lustre: lustre-MDT0000: new disk, initializing [ 8906.079355] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8906.091427] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8909.135378] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 8916.249135] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8916.333731] Lustre: 298374:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8916.372371] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8916.374824] Lustre: Skipped 1 previous similar message [ 8916.427471] Lustre: lustre-MDT0001: new disk, initializing [ 8916.490853] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8916.509905] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8916.514514] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8919.390854] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 8925.123804] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8928.319825] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8932.477765] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8932.661657] Lustre: lustre-OST0000: new disk, initializing [ 8932.664820] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8932.669213] Lustre: 300343:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8932.743219] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8936.807502] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 8938.020287] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8938.030443] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8938.116369] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8943.117295] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8948.816988] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8961.791791] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8970.721522] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9014.752686] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9014.757992] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9014.778961] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9015.268627] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9015.284338] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9015.306753] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9015.311883] Lustre: Skipped 1 previous similar message [ 9017.362278] Lustre: server umount lustre-OST0000 complete [ 9020.388315] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9020.392819] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9020.401665] Lustre: Skipped 1 previous similar message [ 9025.504612] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9025.507809] Lustre: Skipped 2 previous similar messages [ 9026.513601] Lustre: server umount lustre-MDT0000 complete [ 9029.199760] LustreError: 298367:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789306393 with bad export cookie 16336878405733424316 [ 9029.202231] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9029.204914] LustreError: 298367:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9029.442420] Lustre: server umount lustre-MDT0001 complete [ 9034.534437] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9036.936555] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 9039.821809] Lustre: server umount MGS complete [ 9046.474961] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9046.736862] LustreError: 304507:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9046.778349] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9049.600740] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 9052.129662] LustreError: 304494:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9052.139659] LustreError: 304494:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9055.864100] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9056.155894] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9059.276154] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 9063.749443] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9067.123721] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9071.130283] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9071.327198] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9074.623813] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 9078.087968] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9102.308116] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9102.308861] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9102.309350] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9102.317606] Lustre: Skipped 1 previous similar message [ 9105.570404] Lustre: server umount lustre-OST0000 complete [ 9107.427069] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9107.427126] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9107.444076] Lustre: Skipped 1 previous similar message [ 9112.544346] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9112.549602] Lustre: Skipped 4 previous similar messages [ 9113.578166] Lustre: server umount lustre-MDT0000 complete [ 9115.953590] LustreError: 304475:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789306480 with bad export cookie 16336878405733426143 [ 9115.955920] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9115.963856] LustreError: 304475:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9116.175914] Lustre: server umount lustre-MDT0001 complete [ 9121.476590] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [ 9123.105751] Key type lgssc unregistered [ 9123.271229] LNet: 308433:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9123.275777] LNetError: 308433:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9123.289404] LNet: Removed LNI 192.168.201.155@tcp [ 9123.736201] Key type .llcrypt unregistered [ 9123.737641] Key type ._llcrypt unregistered [ 9134.539132] Key type ._llcrypt registered [ 9134.540206] Key type .llcrypt registered [ 9134.595255] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_hostid [ 9143.900953] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 9144.597087] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9144.620038] alg: No test for adler32 (adler32-zlib) [ 9145.615289] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [ 9145.762165] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [ 9147.408129] Key type lgssc registered [ 9148.120659] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9162.630834] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 9171.215336] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9171.243601] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9172.393327] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9172.412403] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9172.455371] Lustre: lustre-MDT0000: new disk, initializing [ 9172.499437] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9172.512564] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9174.461041] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 9180.088750] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9180.255774] Lustre: lustre-OST0000: new disk, initializing [ 9180.258141] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9180.265790] Lustre: Skipped 1 previous similar message [ 9180.271235] Lustre: 312456:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9180.312119] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9183.531947] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 9186.861215] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9186.871528] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9186.895702] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9644.909510] Lustre: server umount lustre-MDT0000 complete [ 9646.883629] LustreError: 313370:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789307011 with bad export cookie 1929907297968213309 [ 9646.888905] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9657.439191] Lustre: server umount lustre-OST0000 complete [ 9672.016903] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_hostid [ 9677.737394] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 9696.361914] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9696.497962] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9696.525963] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9696.576992] Lustre: lustre-MDT0000: new disk, initializing [ 9696.653575] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9696.661903] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9699.170428] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 9705.673947] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9705.746328] Lustre: 316726:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9705.791835] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9705.793849] Lustre: Skipped 1 previous similar message [ 9705.837265] Lustre: lustre-MDT0001: new disk, initializing [ 9705.883463] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9705.897115] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9705.901508] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9708.868333] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 9713.898864] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9716.434961] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9719.526061] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9719.652557] Lustre: lustre-OST0000: new disk, initializing [ 9719.655619] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9719.659966] Lustre: 318690:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9719.709590] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9722.619326] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 9726.465387] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9726.470393] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9726.514581] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9727.834868] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9729.910824] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9730.023462] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9732.181960] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9732.292855] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9736.675440] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9736.675712] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9736.678590] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9736.684975] Lustre: Skipped 1 previous similar message [ 9739.383479] Lustre: server umount lustre-OST0000 complete [ 9741.793118] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9741.799430] LustreError: Skipped 1 previous similar message [ 9741.801828] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9741.805049] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9741.810591] Lustre: Skipped 1 previous similar message [ 9741.821270] Lustre: Skipped 2 previous similar messages [ 9746.916849] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9746.921465] Lustre: Skipped 1 previous similar message [ 9747.638057] Lustre: server umount lustre-MDT0000 complete [ 9749.483169] LustreError: 318688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789307113 with bad export cookie 1929907297971154100 [ 9749.485787] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9749.488345] LustreError: 318688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9749.646100] Lustre: server umount lustre-MDT0001 complete [ 9751.329403] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 9752.120537] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 09:45:15 (1789307115) [ 9755.119983] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9755.373928] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9757.182557] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 9760.736175] LustreError: 321001:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9760.741274] LustreError: 321001:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9761.053058] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9762.914521] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 9766.405504] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9769.038514] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9772.828330] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9773.032602] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9773.039707] Lustre: Skipped 1 previous similar message [ 9775.881564] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 9779.567952] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9782.625899] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9782.697307] Lustre: lustre-OST0001: new disk, initializing [ 9782.699580] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9782.703791] Lustre: 323834:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9785.306759] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 9788.938242] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9788.943968] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9788.961618] LustreError: 324463:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9788.966294] LustreError: 324463:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9789.986604] LustreError: 324455:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9789.992285] LustreError: 324455:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9790.226817] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9790.319631] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9791.010503] LustreError: 324463:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9791.015012] LustreError: 324463:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9792.883737] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff970658f4b800.ost_server_uuid 50 [ 9793.059037] LustreError: 324455:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9793.063689] LustreError: 324455:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 9793.628519] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff970658f4b800.ost_server_uuid in FULL state after 0 sec [ 9796.028145] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff970658f4b800.ost_server_uuid 50 [ 9796.718643] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff970658f4b800.ost_server_uuid in FULL state after 0 sec [ 9797.156407] LustreError: 324455:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9797.162435] LustreError: 324455:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 9798.190873] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9809.381454] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9809.387552] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9814.054424] Lustre: lustre-OST0001: Not available for connect from 192.168.201.55@tcp (stopping) [ 9814.058701] Lustre: Skipped 1 previous similar message [ 9815.249029] Lustre: server umount lustre-OST0001 complete [ 9817.567988] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9817.577238] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9817.582173] Lustre: Skipped 1 previous similar message [ 9819.623483] LustreError: 322823:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9819.632931] LustreError: 322823:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9823.474229] Lustre: server umount lustre-OST0000 complete [ 9827.809630] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9827.819234] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9827.825098] Lustre: Skipped 1 previous similar message [ 9827.829733] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9827.837512] Lustre: Skipped 5 previous similar messages [ 9831.440131] Lustre: server umount lustre-MDT0000 complete [ 9833.344858] LustreError: 320982:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789307197 with bad export cookie 1929907297971154982 [ 9833.347392] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9833.356644] LustreError: 320982:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9833.519892] Lustre: server umount lustre-MDT0001 complete [ 9838.810156] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [ 9840.305990] Key type lgssc unregistered [ 9840.454159] LNet: 326515:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9840.461287] LNetError: 326515:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9840.472342] LNet: Removed LNI 192.168.201.155@tcp [ 9840.859630] Key type .llcrypt unregistered [ 9840.861200] Key type ._llcrypt unregistered [ 9854.433987] Key type ._llcrypt registered [ 9854.436045] Key type .llcrypt registered [ 9854.511223] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [ 9855.164900] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9855.173830] alg: No test for adler32 (adler32-zlib) [ 9856.131545] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [ 9856.310039] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [ 9857.943181] Key type lgssc registered [ 9858.566381] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9862.531552] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9862.540503] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9863.754861] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9865.603067] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 9869.568521] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9872.865867] LustreError: 327907:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9872.874973] LustreError: 327907:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9873.750452] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9873.975180] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9876.215254] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 9881.499453] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9881.761426] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9884.843026] LustreError: 329585:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9885.445763] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 9887.910245] LustreError: 329585:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9887.925692] LustreError: 329585:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9887.928375] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 9890.002105] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9893.350219] LustreError: 330123:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9893.357649] LustreError: 330123:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9893.654535] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9893.773729] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9896.708065] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [ 9898.987835] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 9899.965036] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9903.500528] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 9903.602073] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9996.258289] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9996.258396] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9996.261569] LustreError: Skipped 1 previous similar message [ 9996.262171] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9996.265839] Lustre: Skipped 1 previous similar message [ 9997.457639] Lustre: server umount lustre-OST0001 complete [10000.865409] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10000.876159] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10000.892287] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10000.896347] Lustre: Skipped 1 previous similar message [10001.378962] LustreError: 329587:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10001.394987] LustreError: 329587:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10006.497307] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10006.505233] Lustre: Skipped 2 previous similar messages [10006.788147] Lustre: server umount lustre-OST0000 complete [10011.617737] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10011.622435] Lustre: Skipped 1 previous similar message [10011.628709] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10011.635185] Lustre: Skipped 1 previous similar message [10016.738564] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10016.743428] Lustre: Skipped 1 previous similar message [10022.879164] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10022.984462] Lustre: server umount lustre-MDT0000 complete [10024.980245] LustreError: 330603:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789307389 with bad export cookie 3599234539218265575 [10024.983849] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10024.989458] LustreError: 330603:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10025.008166] LustreError: 327222:0:(import.c:717:ptlrpc_connect_import_locked()) already connecting [10025.213991] Lustre: server umount lustre-MDT0001 complete [10031.038533] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [10032.644737] Key type lgssc unregistered [10032.859616] LNet: 332845:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10032.864920] LNetError: 332845:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10032.878267] LNet: Removed LNI 192.168.201.155@tcp [10033.378124] Key type .llcrypt unregistered [10033.379845] Key type ._llcrypt unregistered [10059.697674] Key type ._llcrypt registered [10059.699442] Key type .llcrypt registered [10059.772712] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [10060.230261] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10060.306816] alg: No test for adler32 (adler32-zlib) [10061.236444] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [10061.426676] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [10063.024420] Key type lgssc registered [10063.698549] Lustre: Echo OBD driver; http://www.lustre.org/ [10068.625990] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10068.635339] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10069.845449] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10071.666311] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [10076.527579] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10076.740910] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10078.505175] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [10079.743299] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10083.116522] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10083.366587] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10086.205847] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [10086.445424] LustreError: 336594:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10086.466962] LustreError: 336594:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10090.672529] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10090.845395] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10094.080071] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [10094.962080] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10094.967749] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10100.406810] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10105.813682] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10112.479778] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10112.484489] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10112.492165] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10115.556346] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10115.558026] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10115.564292] Lustre: Skipped 2 previous similar messages [10115.568311] Lustre: Skipped 2 previous similar messages [10117.486842] Lustre: server umount lustre-MDT0000 complete [10119.386578] LustreError: 335845:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789307483 with bad export cookie 17547406385744754727 [10119.388842] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10119.391408] LustreError: 335845:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10119.576723] Lustre: server umount lustre-MDT0001 complete [10131.800510] Lustre: server umount lustre-OST0000 complete [10144.232858] Lustre: server umount lustre-OST0001 complete [10151.120160] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [10155.639053] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10155.833992] LustreError: 340365:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10155.870267] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10157.821477] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [10161.120266] LustreError: 340366:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10161.773031] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10163.938988] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [10165.483959] Lustre: 341505:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10168.550578] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10168.750206] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10168.754737] Lustre: Skipped 1 previous similar message [10171.822216] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [10172.835184] LustreError: 341859:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10176.215425] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10179.511439] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [10180.487079] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [10181.489773] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [10183.756538] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10185.335374] Lustre: 343371:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10186.723250] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10186.727604] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10186.728454] Lustre: Skipped 2 previous similar messages [10186.733538] Lustre: Skipped 2 previous similar messages [10191.841099] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10191.843936] Lustre: Skipped 3 previous similar messages [10192.811120] Lustre: server umount lustre-MDT0000 complete [10194.289555] LustreError: 340347:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789307558 with bad export cookie 17547406385744756414 [10194.294341] LustreError: 340347:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10194.294816] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10194.443297] Lustre: server umount lustre-MDT0001 complete [10206.212878] Lustre: server umount lustre-OST0000 complete [10217.168218] Lustre: server umount lustre-OST0001 complete [10218.784818] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [10219.898796] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [10220.727159] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 09:53:04 (1789307584) [10230.625944] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [10234.255513] loop0: detected capacity change from 0 to 1099511627776 [10234.415425] SGI XFS with ACLs, security attributes, quota, no debug enabled [10234.421277] *** VALIDATE xfs *** [10234.428641] XFS (loop0): Mounting V5 Filesystem [10234.448067] XFS (loop0): Ending clean mount [10234.977438] loop1: detected capacity change from 0 to 18691697672192 [10257.998862] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_hostid [10261.843206] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [10276.920211] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10277.019935] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10277.033794] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10277.075807] Lustre: lustre-MDT0000: new disk, initializing [10277.109127] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10277.112041] Lustre: Skipped 1 previous similar message [10277.120446] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10278.860864] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [10283.756035] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10283.808366] Lustre: 348484:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10283.834103] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10283.836531] Lustre: Skipped 1 previous similar message [10283.875805] Lustre: lustre-MDT0001: new disk, initializing [10283.934791] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10283.938283] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10285.921986] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [10290.595750] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10292.785714] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10295.410836] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10295.524403] Lustre: lustre-OST0000: new disk, initializing [10295.526848] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10295.530495] Lustre: 350450:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10296.644376] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10296.648566] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10296.697917] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10297.823092] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [10302.020432] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10303.555901] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10303.623806] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10305.130198] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10305.207745] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10307.040866] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10307.042639] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10307.043478] LustreError: Skipped 1 previous similar message [10307.047828] Lustre: Skipped 1 previous similar message [10307.048634] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10307.053041] Lustre: Skipped 1 previous similar message [10317.285430] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10317.290383] Lustre: Skipped 3 previous similar messages [10320.351125] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10320.421463] Lustre: server umount lustre-OST0000 complete [10322.401886] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10322.406823] Lustre: Skipped 2 previous similar messages [10336.223124] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10336.290677] Lustre: server umount lustre-MDT0000 complete [10337.734320] LustreError: 348475:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789307701 with bad export cookie 17547406385744757947 [10337.735723] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10337.741901] LustreError: 348475:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10337.867164] Lustre: server umount lustre-MDT0001 complete [10340.913433] XFS (loop0): Unmounting Filesystem [10342.903955] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 09:55:06 (1789307706) [10345.470037] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10345.643721] LustreError: 353009:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10345.676029] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10345.680728] Lustre: Skipped 2 previous similar messages [10347.343709] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [10350.715212] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10352.575038] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [10355.505812] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10357.596395] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10360.112773] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10362.353633] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [10365.190612] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10375.651288] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10375.653285] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10375.657026] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10375.662047] Lustre: Skipped 1 previous similar message [10375.663970] Lustre: Skipped 6 previous similar messages [10379.865595] Lustre: server umount lustre-OST0000 complete [10381.792765] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10387.477839] Lustre: server umount lustre-MDT0000 complete [10388.895192] LustreError: 353745:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789307753 with bad export cookie 17547406385744758829 [10388.897354] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10388.899176] LustreError: 353745:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10389.018461] Lustre: server umount lustre-MDT0001 complete [10392.814278] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [10393.902741] Key type lgssc unregistered [10394.052591] LNet: 356631:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10394.058266] LNetError: 356631:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10394.071711] LNet: Removed LNI 192.168.201.155@tcp [10394.431188] Key type .llcrypt unregistered [10394.432318] Key type ._llcrypt unregistered [10405.983868] Key type ._llcrypt registered [10405.985028] Key type .llcrypt registered [10406.023594] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [10406.451269] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10406.461785] alg: No test for adler32 (adler32-zlib) [10407.341802] Lustre: Lustre: Build Version: 2.17.57_108_gfed9771 [10407.446988] LNet: Added LNI 192.168.201.155@tcp [8/256/0/180] [10409.031215] Key type lgssc registered [10409.539822] Lustre: Echo OBD driver; http://www.lustre.org/ [10412.594302] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10412.602567] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10413.756112] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10415.363347] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [10418.670453] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10418.828100] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10420.442865] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [10423.196494] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10425.170510] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10427.550259] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10427.677282] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10429.867554] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [10432.660078] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10443.236586] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10443.242063] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10443.244746] Lustre: Skipped 1 previous similar message [10447.125574] Lustre: server umount lustre-MDT0001 complete [10448.352468] LustreError: 358899:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10448.361821] LustreError: 358899:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10451.109466] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10451.190935] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10451.195048] Lustre: Found index 1 for lustre-MDT0001, updating log [10451.212354] Lustre: 358161:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10451.334964] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10452.859390] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [10456.551598] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10460.133797] LustreError: 361950:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10460.138318] LustreError: 361950:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10460.142326] LustreError: 358165:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10460.164365] LustreError: 361958:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.155@tcp: cfg command failed: rc = -17 [10460.168645] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [10460.168645] [10755.614391] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.55@tcp (stopping) [10755.617219] Lustre: Skipped 1 previous similar message [10758.625434] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10758.625866] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10758.630766] Lustre: Skipped 1 previous similar message [10758.635966] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10758.638492] Lustre: Skipped 1 previous similar message [10760.727833] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.55@tcp (stopping) [10761.491181] Lustre: server umount lustre-MDT0001 complete [10763.744319] LustreError: 358172:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10763.750290] LustreError: 358172:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10765.463523] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10765.500848] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10765.502608] Lustre: Found index 1 for lustre-MDT0001, updating log [10765.517333] Lustre: 358162:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10765.638100] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10767.056628] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [10770.919898] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10770.925539] LustreError: 374073:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10770.928585] LustreError: 374073:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10770.931405] LustreError: 358165:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10770.934865] LustreError: 358165:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10770.958322] LustreError: 374076:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [10770.961865] LustreError: 374076:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [10770.964335] LustreError: 374076:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.155@tcp: cfg command failed: rc = -17 [10770.968639] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.155@tcp [10770.968639] [11072.994791] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11072.995800] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11073.000025] Lustre: Skipped 1 previous similar message [11073.004663] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11073.007724] Lustre: Skipped 1 previous similar message [11083.233824] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11083.236212] Lustre: Skipped 5 previous similar messages [11084.255302] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11084.322728] Lustre: server umount lustre-MDT0001 complete [11088.353034] LustreError: 358173:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11088.359794] LustreError: 358173:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11088.643023] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11088.691890] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11088.694658] Lustre: Found index 1 for lustre-MDT0001, updating log [11088.713474] Lustre: 358162:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11088.848457] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [11090.443845] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [11093.991495] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11097.580520] LustreError: 386240:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11097.582991] LustreError: 386240:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11097.587173] LustreError: 358165:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11097.590699] LustreError: 358165:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11097.614598] LustreError: 386243:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [11097.620655] LustreError: 386243:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.155@tcp: cfg command failed: rc = -114 [11097.624321] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [11097.624321] [11406.304209] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11406.307657] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11406.312735] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11406.317185] Lustre: Skipped 3 previous similar messages [11406.321795] Lustre: Skipped 3 previous similar messages [11410.457790] Lustre: server umount lustre-MDT0000 complete [11411.425107] LustreError: 383778:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11411.432033] LustreError: 383778:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11411.777278] LustreError: 358155:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789308775 with bad export cookie 15350209795561209941 [11411.779750] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11411.781713] LustreError: 358155:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11416.544306] LustreError: 358167:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11416.544552] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11426.271346] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11426.363673] Lustre: server umount lustre-MDT0001 complete [11427.889063] Lustre: server umount lustre-OST0000 complete [11431.114154] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_hostid [11435.038626] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [11447.976740] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 10:13:31 (1789308811) [11450.666564] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11450.753933] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11450.767274] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11450.806039] Lustre: lustre-MDT0000: new disk, initializing [11450.839105] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11450.848957] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11452.509323] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [11457.093756] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11457.145557] Lustre: 400301:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11457.171475] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11457.174150] Lustre: Skipped 1 previous similar message [11457.211479] Lustre: lustre-MDT0001: new disk, initializing [11457.251477] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11457.266778] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11457.272340] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11458.817214] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [11462.699718] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11464.638232] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11466.986891] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11467.100163] Lustre: lustre-OST0000: new disk, initializing [11467.102419] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11467.105930] Lustre: 402269:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11467.138232] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11469.334577] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_default_debug -1 all [11473.248338] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11474.426577] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11474.429559] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11474.469301] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11483.104044] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11483.107671] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11483.112689] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11483.114973] Lustre: Skipped 2 previous similar messages [11487.200421] Lustre: server umount lustre-MDT0000 complete [11488.550423] LustreError: 401096:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789308852 with bad export cookie 15350209795561211880 [11488.553304] LustreError: 401109:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [11488.553550] LustreError: MGC192.168.201.155@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11488.554801] LustreError: 401096:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11488.670702] Lustre: server umount lustre-MDT0001 complete [11500.341250] Lustre: server umount lustre-OST0000 complete [11511.485701] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing set_hostid [11515.285307] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing load_modules_local [11527.701798] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 11277 sec ========= 10:14:51 (1789308891) [11528.367235] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 10:14:52 (1789308892) === [11529.677947] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 10:14:53 (1789308893) === [11543.573124] Lustre: DEBUG MARKER: oleg155-server.virtnet: executing unload_modules_local [11544.680061] Key type lgssc unregistered [11544.814316] LNet: 408621:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11544.818289] LNetError: 408621:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11544.828901] LNet: Removed LNI 192.168.201.155@tcp [11545.280840] Key type .llcrypt unregistered [11545.282092] Key type ._llcrypt unregistered