[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 514994491 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.002354] x2apic enabled [ 0.003008] Switched APIC routing to physical x2apic. [ 0.004011] kvm-guest: setup PV IPIs [ 0.006690] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007028] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008013] pid_max: default: 32768 minimum: 301 [ 0.009130] LSM: Security Framework initializing [ 0.010071] Yama: becoming mindful. [ 0.011037] SELinux: Initializing. [ 0.012064] *** VALIDATE selinux *** [ 0.021102] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026231] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027165] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028116] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030004] *** VALIDATE tmpfs *** [ 0.031425] *** VALIDATE proc *** [ 0.032219] *** VALIDATE cgroup *** [ 0.033007] *** VALIDATE cgroup2 *** [ 0.034265] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035159] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037029] Spectre V2 : User space: Vulnerable [ 0.038009] Speculative Store Bypass: Vulnerable [ 0.041301] debug: unmapping init [mem 0xffffffffac459000-0xffffffffac460fff] [ 0.043170] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044681] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045022] ... version: 2 [ 0.046012] ... bit width: 48 [ 0.047014] ... generic registers: 4 [ 0.048010] ... value mask: 0000ffffffffffff [ 0.049012] ... max period: 00007fffffffffff [ 0.050012] ... fixed-purpose events: 3 [ 0.051010] ... event mask: 000000070000000f [ 0.052349] rcu: Hierarchical SRCU implementation. [ 0.054484] smp: Bringing up secondary CPUs ... [ 0.055569] x86: Booting SMP configuration: [ 0.056030] .... node #0, CPUs: #1 #2 #3 [ 0.059052] smp: Brought up 1 node, 4 CPUs [ 0.061011] smpboot: Max logical packages: 1 [ 0.062019] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.142193] node 0 deferred pages initialised in 78ms [ 0.145122] devtmpfs: initialized [ 0.146211] x86/mm: Memory block size: 128MB [ 0.148774] gcov: version magic: 0x41383552 [ 0.149665] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.153080] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.156387] pinctrl core: initialized pinctrl subsystem [ 0.158216] [ 0.158826] ************************************************************* [ 0.161020] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.163010] ** ** [ 0.165021] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.168014] ** ** [ 0.170025] ** This means that this kernel is built to expose internal ** [ 0.173011] ** IOMMU data structures, which may compromise security on ** [ 0.175015] ** your system. ** [ 0.177017] ** ** [ 0.179013] ** If you see this message and you are not debugging the ** [ 0.181021] ** kernel, report this immediately to your vendor! ** [ 0.183034] ** ** [ 0.185019] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.187030] ************************************************************* [ 0.190090] NET: Registered protocol family 16 [ 0.192519] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.194049] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.197046] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.200108] cpuidle: using governor menu [ 0.201727] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.204418] PCI: Using configuration type 1 for base access [ 0.206187] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.215176] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.218029] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.221073] cryptd: max_cpu_qlen set to 1000 [ 0.224201] ACPI: Added _OSI(Module Device) [ 0.226014] ACPI: Added _OSI(Processor Device) [ 0.227008] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.228014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.232406] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.237478] ACPI: Interpreter enabled [ 0.239060] ACPI: PM: (supports S0 S3 S4 S5) [ 0.240011] ACPI: Using IOAPIC for interrupt routing [ 0.242106] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.245388] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.255336] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.257052] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.260020] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.263080] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.268129] acpiphp: Slot [2] registered [ 0.269085] acpiphp: Slot [5] registered [ 0.270074] acpiphp: Slot [6] registered [ 0.271073] acpiphp: Slot [7] registered [ 0.272122] acpiphp: Slot [8] registered [ 0.273095] acpiphp: Slot [9] registered [ 0.275083] acpiphp: Slot [10] registered [ 0.276102] acpiphp: Slot [3] registered [ 0.277105] acpiphp: Slot [4] registered [ 0.279109] acpiphp: Slot [11] registered [ 0.280144] acpiphp: Slot [12] registered [ 0.281100] acpiphp: Slot [13] registered [ 0.283097] acpiphp: Slot [14] registered [ 0.284108] acpiphp: Slot [15] registered [ 0.285086] acpiphp: Slot [16] registered [ 0.286088] acpiphp: Slot [17] registered [ 0.288102] acpiphp: Slot [18] registered [ 0.289126] acpiphp: Slot [19] registered [ 0.291141] acpiphp: Slot [20] registered [ 0.292094] acpiphp: Slot [21] registered [ 0.294095] acpiphp: Slot [22] registered [ 0.295103] acpiphp: Slot [23] registered [ 0.296082] acpiphp: Slot [24] registered [ 0.298175] acpiphp: Slot [25] registered [ 0.299072] acpiphp: Slot [26] registered [ 0.301117] acpiphp: Slot [27] registered [ 0.302230] acpiphp: Slot [28] registered [ 0.304095] acpiphp: Slot [29] registered [ 0.305088] acpiphp: Slot [30] registered [ 0.307090] acpiphp: Slot [31] registered [ 0.308097] PCI host bridge to bus 0000:00 [ 0.310020] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.312024] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.315024] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.317033] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.320026] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.323028] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.325188] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.327940] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.331399] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.340796] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.346067] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.350032] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.351022] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.353024] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.356695] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.359665] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.361061] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.364363] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.370018] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.382017] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.387016] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.392982] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.403021] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.411022] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.437021] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.451472] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.459029] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.469028] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.484028] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.496428] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.504017] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.512013] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.530016] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.541000] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.550023] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.561025] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.584039] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.596176] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.602019] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.611021] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.632022] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.647036] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.679039] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.706024] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.781041] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.792153] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.795360] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.797379] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.799384] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.803313] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.808115] iommu: Default domain type: Passthrough [ 0.809539] SCSI subsystem initialized [ 0.811198] ACPI: bus type USB registered [ 0.812100] usbcore: registered new interface driver usbfs [ 0.814068] usbcore: registered new interface driver hub [ 0.816078] usbcore: registered new device driver usb [ 0.818180] pps_core: LinuxPPS API ver. 1 registered [ 0.820021] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.823062] PTP clock support registered [ 0.825111] EDAC MC: Ver: 3.0.0 [ 0.827252] PCI: Using ACPI for IRQ routing [ 0.828706] NetLabel: Initializing [ 0.830015] NetLabel: domain hash size = 128 [ 0.832013] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.834081] NetLabel: unlabeled traffic allowed by default [ 0.837091] vgaarb: loaded [ 0.838292] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.840013] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.847908] clocksource: Switched to clocksource kvm-clock [ 0.962636] VFS: Disk quotas dquot_6.6.0 [ 0.964153] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.966685] *** VALIDATE ramfs *** [ 0.968080] *** VALIDATE hugetlbfs *** [ 0.969654] pnp: PnP ACPI init [ 0.972040] pnp: PnP ACPI: found 6 devices [ 1.005397] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.008897] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.011058] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.013374] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.016521] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.018785] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.021335] NET: Registered protocol family 2 [ 1.023707] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.028713] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.032366] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.037654] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.041437] TCP: Hash tables configured (established 65536 bind 65536) [ 1.044716] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.048465] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.051360] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.054341] NET: Registered protocol family 1 [ 1.056972] RPC: Registered named UNIX socket transport module. [ 1.059334] RPC: Registered udp transport module. [ 1.062384] RPC: Registered tcp transport module. [ 1.064041] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.066611] NET: Registered protocol family 44 [ 1.068433] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.071380] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.073236] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.075372] PCI: CLS 0 bytes, default 64 [ 1.076838] Unpacking initramfs... [ 2.462792] debug: unmapping init [mem 0xffff91117cc54000-0xffff91117ffbffff] [ 2.466685] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.468924] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.471928] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.984808] Initialise system trusted keyrings [ 2.988266] Key type blacklist registered [ 2.990377] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.001718] zbud: loaded [ 3.006274] *** VALIDATE nfs *** [ 3.007672] *** VALIDATE nfs4 *** [ 3.009255] pstore: using deflate compression [ 3.012681] Platform Keyring initialized [ 3.124971] NET: Registered protocol family 38 [ 3.127023] Key type asymmetric registered [ 3.128724] Asymmetric key parser 'x509' registered [ 3.130680] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.133240] io scheduler mq-deadline registered [ 3.134618] io scheduler kyber registered [ 3.135818] io scheduler bfq registered [ 3.137673] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.140316] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.142796] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.145659] ACPI: Power Button [PWRF] [ 3.150621] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.156378] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.166856] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.174880] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.193064] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.223237] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.252717] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.257951] Non-volatile memory driver v1.3 [ 3.259650] Linux agpgart interface v0.103 [ 3.285078] virtio_blk virtio1: [vda] 150672 512-byte logical blocks (77.1 MB/73.6 MiB) [ 3.287341] vda: detected capacity change from 0 to 77144064 [ 3.300641] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.302544] vdb: detected capacity change from 0 to 1073741824 [ 3.315719] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.317544] vdc: detected capacity change from 0 to 2621440000 [ 3.331813] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.335190] vdd: detected capacity change from 0 to 2621440000 [ 3.351553] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.355267] vde: detected capacity change from 0 to 4294967296 [ 3.371513] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.373435] vdf: detected capacity change from 0 to 4294967296 [ 3.379422] libphy: Fixed MDIO Bus: probed [ 3.385919] usbcore: registered new interface driver usbserial_generic [ 3.388173] usbserial: USB Serial support registered for generic [ 3.390193] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.393871] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.395544] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.397646] mousedev: PS/2 mouse device common for all mice [ 3.400246] rtc_cmos 00:05: RTC can wake from S4 [ 3.403286] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.407170] rtc_cmos 00:05: registered as rtc0 [ 3.408401] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.409826] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.410641] intel_pstate: CPU model not supported [ 3.416772] hid: raw HID events driver (C) Jiri Kosina [ 3.416893] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.418404] usbcore: registered new interface driver usbhid [ 3.423572] usbhid: USB HID core driver [ 3.425270] drop_monitor: Initializing network drop monitor service [ 3.427721] Initializing XFRM netlink socket [ 3.429965] NET: Registered protocol family 10 [ 3.432602] Segment Routing with IPv6 [ 3.433757] NET: Registered protocol family 17 [ 3.435082] mpls_gso: MPLS GSO support [ 3.439956] RAS: Correctable Errors collector initialized. [ 3.442327] AVX version of gcm_enc/dec engaged. [ 3.443657] AES CTR mode by8 optimization enabled [ 3.507069] sched_clock: Marking stable (3507039785, 0)->(4369490483, -862450698) [ 3.510679] registered taskstats version 1 [ 3.512300] Loading compiled-in X.509 certificates [ 3.513600] zswap: loaded using pool lzo/zbud [ 3.533417] Key type big_key registered [ 3.545612] Key type encrypted registered [ 3.547192] ima: No TPM chip found, activating TPM-bypass! [ 3.549349] ima: Allocated hash algorithm: sha1 [ 3.551035] ima: No architecture policies found [ 3.552870] evm: Initialising EVM extended attributes: [ 3.554548] evm: security.selinux [ 3.555819] evm: security.ima [ 3.557104] evm: security.capability [ 3.558523] evm: HMAC attrs: 0x1 [ 3.561581] rtc_cmos 00:05: setting system clock to 2026-09-30 03:35:09 UTC (1790739309) [ 3.568433] debug: unmapping init [mem 0xffffffffad403000-0xffffffffad5fffff] [ 3.571602] debug: unmapping init [mem 0xffffffffac182000-0xffffffffac458fff] [ 3.580071] Write protecting the kernel read-only data: 28672k [ 3.583511] debug: unmapping init [mem 0xffffffffaa803000-0xffffffffaa9fffff] [ 3.586492] debug: unmapping init [mem 0xffffffffab114000-0xffffffffab1fffff] [ 3.620515] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.628880] systemd[1]: Detected virtualization kvm. [ 3.630639] systemd[1]: Detected architecture x86-64. [ 3.632889] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.662898] systemd[1]: No hostname configured. [ 3.665087] systemd[1]: Set hostname to . [ 3.667125] random: systemd: uninitialized urandom read (16 bytes read) [ 3.669613] systemd[1]: Initializing machine ID from random generator. [ 3.798498] random: systemd: uninitialized urandom read (16 bytes read) [ 3.801460] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 3.805886] random: systemd: uninitialized urandom read (16 bytes read) [ 3.807775] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 3.810796] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket. [ OK ] Reached target Slices. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Initrd Root Device. Starting Create Volatile Files and Directories... Starting Apply Kernel Variables... Starting Setup Virtual Console... [ OK ] Reached target Timers. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.406342] device-mapper: uevent: version 1.0.3 [ 4.409150] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.072287] virtio_net virtio0 ens2: renamed from eth0 [ 5.083676] random: fast init done Starting dracut initqueue hook... [ 5.177217] scsi host0: ata_piix [ 5.179295] scsi host1: ata_piix [ 5.180763] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.183249] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 8.742414] dracut-initqueue[581]: RTNETLINK answers: File exists [ 10.063987] random: crng init done [ 10.065448] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.336969] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Slices. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.404909] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.653400] SELinux: Disabled at runtime. [ 11.715361] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.724490] systemd[1]: Detected virtualization kvm. [ 11.726299] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.219876] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.223744] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.231234] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.236062] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.239597] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.249429] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.279604] systemd[1]: Listening on Process Core Dump Socket. [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-serial\x2dgetty.slice. Starting Remount Root and Kernel File Systems... Mounting Huge Pages File System... Mounting Kernel Debug File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. Starting Apply Kernel Variables... [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice User and Session Slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Created slice system-sshd\x2dkeygen.slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-getty.slice. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on udev Control Socket. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. Activating swap /dev/disk/by-label/SWAP... Mounting POSIX Message Queue File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Slices. [ OK ] Reached target Paths. [ 12.452621] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 12.711575] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.000267] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.044801] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.110814] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.133624] EDAC sbridge: Ver: 1.1.2 [ 14.743919] Key type dns_resolver registered [ 15.057720] NFS: Registering the id_resolver key type [ 15.059389] Key type id_resolver registered [ 15.060530] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Rebuild Dynamic Linker Cache... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Reached target sshd-keygen.target. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Started OpenSSH server daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg247-server login: [ 40.694673] libcfs: loading out-of-tree module taints kernel. [ 40.715818] Key type ._llcrypt registered [ 40.717185] Key type .llcrypt registered [ 40.770319] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 50.988784] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 52.762512] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 52.780884] alg: No test for adler32 (adler32-zlib) [ 54.356754] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [ 55.131485] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 56.958638] Key type lgssc registered [ 58.720376] Lustre: Echo OBD driver; http://www.lustre.org/ [ 78.686443] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 120.437949] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 134.685905] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 134.737661] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 136.162646] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 136.190655] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 136.306374] Lustre: lustre-MDT0000: new disk, initializing [ 136.481427] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 136.511820] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 141.680455] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 156.200776] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 156.320948] Lustre: 6486:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 156.352707] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 156.356263] Lustre: Skipped 1 previous similar message [ 156.469179] Lustre: lustre-MDT0001: new disk, initializing [ 156.573924] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 156.609511] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 156.620655] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 161.166785] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 166.466595] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 177.422835] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 177.631932] Lustre: lustre-OST0000: new disk, initializing [ 177.635875] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 177.642289] Lustre: 8421:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 177.739129] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 181.451950] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 181.454842] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 181.526183] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 184.861362] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 200.611098] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 200.783129] Lustre: lustre-OST0001: new disk, initializing [ 200.787880] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 200.795062] Lustre: 9495:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 200.871377] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 208.068479] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 211.029168] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 211.038774] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 211.171124] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 220.171042] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 227.314606] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 234.383209] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing check_logdir /tmp/testlogs/ [ 239.354370] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing yml_node [ 243.543885] Lustre: DEBUG MARKER: Client: 2.17.59.3 [ 247.085342] Lustre: DEBUG MARKER: MDS: 2.17.59.3 [ 250.046315] Lustre: DEBUG MARKER: OSS: 2.17.59.3 [ 251.887547] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Sep 29 23:39:16 EDT 2026 [ 257.485066] hrtimer: interrupt took 3283556 ns [ 269.821547] Lustre: DEBUG MARKER: only running tests: 76-120 [ 271.666113] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 273.092228] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 289.759982] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 289.763769] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 289.777357] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 293.344644] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 293.346819] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 293.349174] Lustre: Skipped 1 previous similar message [ 293.355511] Lustre: Skipped 1 previous similar message [ 298.467683] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 298.485055] Lustre: Skipped 5 previous similar messages [ 303.594953] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 304.095162] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 304.479184] Lustre: server umount lustre-MDT0000 complete [ 309.506534] LustreError: 6477:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790739615 with bad export cookie 2829598042118471717 [ 309.509270] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 309.538971] LustreError: 6477:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 313.823892] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 313.824780] LustreError: 6491:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 313.825249] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 313.825253] Lustre: Skipped 3 previous similar messages [ 313.852551] Lustre: Skipped 2 previous similar messages [ 313.896919] LustreError: 6491:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [ 314.848453] LustreError: 6492:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 314.867159] LustreError: 6492:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 316.066476] Lustre: server umount lustre-MDT0001 complete [ 321.346804] Lustre: server umount lustre-OST0000 complete [ 327.074938] Lustre: server umount lustre-OST0001 complete [ 338.514545] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 353.515679] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 392.358814] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 392.603863] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 392.669751] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 392.777833] Lustre: lustre-MDT0000: new disk, initializing [ 392.874898] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 392.892640] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 398.033829] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 409.370723] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 409.510332] Lustre: 15119:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 409.578230] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 409.581688] Lustre: Skipped 1 previous similar message [ 409.649125] Lustre: lustre-MDT0001: new disk, initializing [ 409.740205] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 409.777147] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 409.798250] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 414.677028] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 423.985287] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 430.429303] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 437.853590] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 438.189723] Lustre: lustre-OST0000: new disk, initializing [ 438.194466] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 438.200957] Lustre: 17087:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 438.352654] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 439.383529] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 439.408769] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 439.580126] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 445.290693] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 455.127273] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 459.590902] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 459.792447] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 465.540642] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 465.803569] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 470.512928] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 470.514506] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 470.528104] LustreError: Skipped 1 previous similar message [ 470.561908] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 470.561916] Lustre: Skipped 3 previous similar messages [ 474.797208] Lustre: server umount lustre-OST0000 complete [ 480.740264] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 480.760339] Lustre: Skipped 2 previous similar messages [ 485.374186] Lustre: server umount lustre-MDT0000 complete [ 485.859248] LustreError: 16000:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 485.878271] LustreError: 16000:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 488.942028] LustreError: 15111:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790739794 with bad export cookie 2829598042118473243 [ 488.948951] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 488.953346] LustreError: 15111:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 489.280128] Lustre: server umount lustre-MDT0001 complete [ 494.366356] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 23:43:19 (1790739799) [ 501.479042] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 501.862175] LustreError: 19353:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 501.948269] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 506.358472] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 507.383028] LustreError: 19354:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 515.220369] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 520.999454] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 529.152491] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 535.161676] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 541.977595] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 549.635418] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 558.697451] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 565.711844] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 571.872983] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 571.878911] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 571.888972] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 571.894674] Lustre: Skipped 3 previous similar messages [ 572.199827] Lustre: server umount lustre-MDT0000 complete [ 575.980592] LustreError: 20307:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 576.001245] LustreError: 20307:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 576.348243] LustreError: 19334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790739882 with bad export cookie 2829598042118474132 [ 576.351509] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 576.354995] LustreError: 19334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 576.870643] Lustre: server umount lustre-MDT0001 complete [ 592.354434] Lustre: server umount lustre-OST0000 complete [ 619.314615] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 630.238970] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 630.727572] LustreError: 23736:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 630.831332] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 630.836973] Lustre: Skipped 2 previous similar messages [ 635.870854] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 645.539613] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 651.149937] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 654.205463] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 661.917446] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 669.896950] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 679.910269] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 680.060658] Lustre: lustre-OST0001: new disk, initializing [ 680.064330] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 680.069483] Lustre: 26052:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 681.428996] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 681.442423] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 681.593213] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 687.602475] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 698.171248] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 702.353417] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 717.282702] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 717.288921] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 717.300939] Lustre: Skipped 4 previous similar messages [ 717.323393] Lustre: Skipped 3 previous similar messages [ 723.688184] Lustre: server umount lustre-MDT0000 complete [ 727.260117] LustreError: 27020:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790740033 with bad export cookie 2829598042118474979 [ 727.268943] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 727.277152] LustreError: 27020:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 727.522054] LustreError: 23732:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 727.528984] LustreError: 23732:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 727.628832] Lustre: server umount lustre-MDT0001 complete [ 734.194314] Lustre: server umount lustre-OST0000 complete [ 738.604570] Lustre: server umount lustre-OST0001 complete [ 756.297479] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 768.613777] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 769.292912] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 769.300326] Lustre: Skipped 3 previous similar messages [ 774.355952] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 783.319761] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 788.056972] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 791.641741] Lustre: 30217:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 799.994512] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 808.845844] LustreError: 30571:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 808.881766] LustreError: 30571:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [ 809.729349] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 820.482580] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 828.234446] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 836.977402] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 840.329994] Lustre: 32095:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 850.410450] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 850.420970] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 850.436878] Lustre: Skipped 3 previous similar messages [ 850.444113] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 850.447426] Lustre: Skipped 6 previous similar messages [ 853.848719] Lustre: server umount lustre-MDT0000 complete [ 857.900806] LustreError: 29051:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790740163 with bad export cookie 2829598042118476274 [ 857.910659] LustreError: 29051:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 857.922798] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 858.310241] Lustre: server umount lustre-MDT0001 complete [ 872.369443] Lustre: server umount lustre-OST0000 complete [ 885.545466] Lustre: server umount lustre-OST0001 complete [ 897.226542] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 23:50:02 (1790740202) [ 925.565876] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 936.044907] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 941.539295] LustreError: 34904:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 941.556251] LustreError: 34904:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 10 previous similar messages [ 941.761335] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 952.201108] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 957.732753] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 960.663738] Lustre: 36052:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 967.853181] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 975.847779] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 984.855358] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 992.376569] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1001.367602] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1005.355901] Lustre: 37935:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1013.737067] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1013.751964] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1013.787694] Lustre: Skipped 3 previous similar messages [ 1013.799938] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1013.812669] Lustre: Skipped 4 previous similar messages [ 1017.361912] Lustre: server umount lustre-MDT0000 complete [ 1022.135657] LustreError: 34885:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790740328 with bad export cookie 2829598042118477380 [ 1022.147564] LustreError: 34885:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1022.160776] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1022.680191] Lustre: server umount lustre-MDT0001 complete [ 1037.920133] Lustre: server umount lustre-OST0000 complete [ 1041.388100] Lustre: 3618:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790740331/real 1790740331] req@ffff9110c2654a80 x1877726316553344/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790740347 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1042.143165] Lustre: 3616:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790740332/real 1790740332] req@ffff9110c3672680 x1877726316553600/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790740348 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1043.690436] Lustre: server umount lustre-OST0001 complete [ 1055.775156] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 23:52:40 (1790740360) [ 1075.975380] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1089.030054] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1089.646164] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1089.652656] Lustre: Skipped 7 previous similar messages [ 1095.039788] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1104.620310] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1110.102201] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1114.022757] Lustre: 41224:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1122.384112] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1131.117690] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1142.742390] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1150.787386] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1159.773914] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1164.459984] Lustre: 43104:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1165.870903] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1165.876683] Lustre: Skipped 1 previous similar message [ 1171.939973] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1171.946733] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1171.957117] Lustre: Skipped 4 previous similar messages [ 1176.211815] Lustre: server umount lustre-MDT0000 complete [ 1181.539384] LustreError: 40058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790740487 with bad export cookie 2829598042118478486 [ 1181.544961] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1181.547260] LustreError: 40058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 1188.254203] Lustre: server umount lustre-MDT0001 complete [ 1199.252511] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1207.044806] LustreError: 41589:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1207.066526] LustreError: 41589:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 28 previous similar messages [ 1212.673642] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1216.501932] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1223.369649] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1228.987972] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1229.309517] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1229.316457] Lustre: Skipped 1 previous similar message [ 1237.413761] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1239.434176] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1245.758883] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1247.741661] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1254.892355] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1260.533752] Lustre: server umount lustre-MDT0000 complete [ 1264.487490] LustreError: 40058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790740570 with bad export cookie 2829598042118479585 [ 1264.488247] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1264.532176] LustreError: 40058:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1264.832736] Lustre: server umount lustre-MDT0001 complete [ 1278.391359] Lustre: server umount lustre-OST0000 complete [ 1281.503102] Lustre: 3615:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790740571/real 1790740571] req@ffff9110c3639500 x1877726316647808/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790740587 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1283.552762] Lustre: server umount lustre-OST0001 complete [ 1296.644558] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 23:56:41 (1790740601) [ 1320.267862] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1332.705394] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1336.775336] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1348.472126] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1353.015312] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1356.438776] Lustre: 48886:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1365.602519] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1374.131156] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1383.597530] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1390.788871] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1398.987867] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1404.014070] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1415.649903] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1415.667899] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1415.678395] Lustre: Skipped 23 previous similar messages [ 1421.101858] Lustre: server umount lustre-MDT0000 complete [ 1425.381067] LustreError: 47715:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790740731 with bad export cookie 2829598042118480404 [ 1425.383676] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1425.409802] LustreError: 47715:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1425.656499] Lustre: server umount lustre-MDT0001 complete [ 1440.546536] Lustre: server umount lustre-OST0000 complete [ 1454.841401] Lustre: server umount lustre-OST0001 complete [ 1466.352549] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 23:59:31 (1790740771) [ 1469.356156] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1471.254917] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 23:59:36 (1790740776) [ 1484.567820] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 1496.711595] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1523.131028] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1523.467800] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1523.575054] Lustre: lustre-MDT0000: new disk, initializing [ 1523.680790] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1528.281469] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1538.334165] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1546.136720] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1546.474225] Lustre: lustre-OST0000: new disk, initializing [ 1546.476961] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1546.481765] Lustre: Skipped 1 previous similar message [ 1546.490280] Lustre: 54971:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1548.393977] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1548.402685] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1548.436726] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1553.788367] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1564.705854] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1567.142459] Lustre: DEBUG MARKER: create test files [ 1599.468409] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1599.469164] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1599.490215] Lustre: Skipped 14 previous similar messages [ 1613.279183] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1613.805758] Lustre: server umount lustre-OST0000 complete [ 1619.212869] Lustre: server umount lustre-MDT0000 complete [ 1632.510900] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 1635.363368] Key type lgssc unregistered [ 1635.768088] LNet: 56617:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1635.778836] LNetError: 56617:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1635.803472] LNet: Removed LNI 192.168.202.147@tcp [ 1637.235193] Key type .llcrypt unregistered [ 1637.237085] Key type ._llcrypt unregistered [ 1671.373959] Key type ._llcrypt registered [ 1671.377821] Key type .llcrypt registered [ 1671.554199] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1672.725286] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1672.895697] alg: No test for adler32 (adler32-zlib) [ 1674.043436] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [ 1674.288795] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 1676.087526] Key type lgssc registered [ 1677.720918] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1686.924690] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1686.993310] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1688.633851] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1694.365223] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1703.640821] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1711.231930] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1711.700654] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1713.772507] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:97 to 0x240000400:129) [ 1713.866112] Lustre: 59370:0:(osd_internal.h:1462:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1713.873363] Lustre: 59370:0:(osd_handler.c:2097:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1713.882796] Lustre: 59370:0:(osd_handler.c:2104:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1713.890990] Lustre: 59370:0:(osd_handler.c:2114:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1713.898354] Lustre: 59370:0:(osd_handler.c:2121:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1713.908523] Lustre: 59370:0:(osd_handler.c:2128:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1719.296291] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1728.421797] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1734.389186] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1740.706821] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1749.990893] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1750.005130] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1750.052837] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1755.113902] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1755.828589] Lustre: server umount lustre-OST0000 complete [ 1762.111866] Lustre: server umount lustre-MDT0000 complete [ 1780.803023] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 1784.380075] Key type lgssc unregistered [ 1784.951642] LNet: 61001:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1784.961313] LNetError: 61001:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1785.012188] LNet: Removed LNI 192.168.202.147@tcp [ 1786.283559] Key type .llcrypt unregistered [ 1786.285320] Key type ._llcrypt unregistered [ 1823.022512] Key type ._llcrypt registered [ 1823.025953] Key type .llcrypt registered [ 1823.167969] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1824.891430] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1824.931549] alg: No test for adler32 (adler32-zlib) [ 1826.027248] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [ 1826.359094] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 1828.135235] Key type lgssc registered [ 1829.756869] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1839.659347] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 1839.689788] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1841.251187] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1847.633917] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1857.407398] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1864.991711] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1865.397897] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1866.436418] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1866.570753] Lustre: 63848:0:(osd_internal.h:1462:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1866.585736] Lustre: 63848:0:(osd_handler.c:2097:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1866.595520] Lustre: 63848:0:(osd_handler.c:2104:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1866.604562] Lustre: 63848:0:(osd_handler.c:2114:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1866.614405] Lustre: 63848:0:(osd_handler.c:2121:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1866.619417] Lustre: 63848:0:(osd_handler.c:2128:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1873.373122] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1882.166624] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1888.066505] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1928.161785] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1928.164842] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1928.207597] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1932.552089] Lustre: server umount lustre-OST0000 complete [ 1937.280642] Lustre: server umount lustre-MDT0000 complete [ 1951.427153] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 1954.644193] Key type lgssc unregistered [ 1955.029318] LNet: 65433:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1955.037450] LNetError: 65433:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1955.062985] LNet: Removed LNI 192.168.202.147@tcp [ 1956.020656] Key type .llcrypt unregistered [ 1956.022984] Key type ._llcrypt unregistered [ 1975.207461] Key type ._llcrypt registered [ 1975.209215] Key type .llcrypt registered [ 1975.281335] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 1991.160632] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1992.390410] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1992.467372] alg: No test for adler32 (adler32-zlib) [ 1993.632666] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [ 1993.906045] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 1995.639217] Key type lgssc registered [ 1997.003903] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2037.120213] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 2037.173043] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2038.450667] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2038.511942] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2038.664265] Lustre: lustre-MDT0000: new disk, initializing [ 2038.819916] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2038.833965] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2044.369275] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2057.301908] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2057.521129] Lustre: 68600:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2057.588884] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2057.595435] Lustre: Skipped 1 previous similar message [ 2057.694978] Lustre: lustre-MDT0001: new disk, initializing [ 2057.803593] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2057.833981] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2057.843836] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2062.826430] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2075.074652] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2082.015828] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2089.546487] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2090.011856] Lustre: lustre-OST0000: new disk, initializing [ 2090.017541] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2090.026727] Lustre: 70571:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2090.154827] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2095.699740] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2095.716118] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2095.897684] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2096.536946] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2107.111509] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2112.301679] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2112.488876] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2117.035381] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2117.196717] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2121.706222] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2121.707024] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2121.713148] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2121.715369] Lustre: Skipped 1 previous similar message [ 2121.733562] Lustre: Skipped 1 previous similar message [ 2125.668117] Lustre: server umount lustre-OST0000 complete [ 2129.379265] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2129.391119] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2129.405475] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2131.943452] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2131.947616] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2135.527380] Lustre: server umount lustre-MDT0000 complete [ 2140.177819] LustreError: 68590:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790741446 with bad export cookie 15922219114311491974 [ 2140.181274] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2140.197218] LustreError: 68590:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2140.586976] Lustre: server umount lustre-MDT0001 complete [ 2151.491848] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 00:10:56 (1790741456) [ 2166.102942] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 2210.192387] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 2219.099851] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 2260.912420] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2261.228131] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2261.255170] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2261.354259] Lustre: lustre-MDT0000: new disk, initializing [ 2261.457550] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2261.506182] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2266.857839] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2278.082772] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2278.266385] Lustre: 76616:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2278.317833] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2278.321795] Lustre: Skipped 1 previous similar message [ 2278.379639] Lustre: lustre-MDT0001: new disk, initializing [ 2278.479931] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2278.526843] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2278.555480] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2283.972170] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2294.887464] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2301.053747] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2309.047111] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2309.280377] Lustre: lustre-OST0000: new disk, initializing [ 2309.283900] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2309.290443] Lustre: 78582:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2309.368264] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2316.134486] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2317.359651] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2317.366656] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2317.487837] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2326.972238] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2332.280707] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2332.541370] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2337.256986] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2337.478338] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2343.392827] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2343.402393] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2343.402677] LustreError: Skipped 1 previous similar message [ 2343.419730] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2343.423889] Lustre: Skipped 1 previous similar message [ 2345.808595] Lustre: server umount lustre-OST0000 complete [ 2350.050368] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2350.057360] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2350.074886] Lustre: Skipped 1 previous similar message [ 2350.087788] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2358.753890] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2358.765503] Lustre: Skipped 3 previous similar messages [ 2364.383526] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2364.880239] Lustre: server umount lustre-MDT0000 complete [ 2368.993657] LustreError: 76624:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2369.009653] LustreError: 76624:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [ 2369.336146] LustreError: 76610:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790741675 with bad export cookie 15922219114311492856 [ 2369.337647] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2369.351157] LustreError: 76610:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2369.754232] Lustre: server umount lustre-MDT0001 complete [ 2380.464380] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 00:14:45 (1790741685) [ 2388.789901] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2389.611088] LustreError: 80997:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2389.707154] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2395.115302] LustreError: 80998:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2395.394557] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2399.203161] LustreError: 80997:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2404.323872] LustreError: 80998:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2404.786375] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2410.148794] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2417.832441] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2423.747953] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2430.881343] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2431.231076] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2431.238079] Lustre: Skipped 1 previous similar message [ 2437.276505] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2444.124356] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2447.520406] Lustre: 83679:0:(genops.c:1773:obd_export_evict_by_uuid()) MGS: evicting 5f845170-c0a1-4615-a162-0a8891d044d4 at adminstrative request [ 2449.653588] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2449.677791] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2459.113853] Lustre: MGS: Client 5f845170-c0a1-4615-a162-0a8891d044d4 (at 0@lo) reconnecting [ 2459.123071] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xdcf715ecc878a86a to 0xdcf715ecc878ab0a [ 2459.138866] Lustre: MGC192.168.202.147@tcp: Connection restored to 0@lo (at 0@lo) [ 2483.658419] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2483.766400] Lustre: lustre-OST0001: new disk, initializing [ 2483.769578] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2483.773409] Lustre: 84072:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2489.689258] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2490.442910] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2490.451157] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2490.547609] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2501.232118] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2506.225234] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2506.229842] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2506.241809] Lustre: Skipped 1 previous similar message [ 2506.242744] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2506.242748] Lustre: Skipped 2 previous similar messages [ 2506.277620] LustreError: Skipped 1 previous similar message [ 2511.351162] LustreError: 82829:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2511.360636] LustreError: 82829:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2511.397681] Lustre: server umount lustre-OST0001 complete [ 2516.448971] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2516.478377] LustreError: Skipped 1 previous similar message [ 2521.659837] Lustre: server umount lustre-OST0000 complete [ 2528.228234] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2528.244490] Lustre: Skipped 3 previous similar messages [ 2533.222483] Lustre: server umount lustre-MDT0000 complete [ 2536.940579] LustreError: 80993:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2536.954828] LustreError: 80993:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 7 previous similar messages [ 2537.355786] LustreError: 81741:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790741843 with bad export cookie 15922219114311494410 [ 2537.361323] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2537.379382] LustreError: 81741:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2537.620409] Lustre: server umount lustre-MDT0001 complete [ 2546.904436] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 00:17:32 (1790741852) [ 2549.195235] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2551.502934] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 00:17:36 (1790741856) [ 2553.544938] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2555.572798] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 00:17:40 (1790741860) [ 2557.185361] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2559.033625] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 00:17:44 (1790741864) [ 2560.677652] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2562.863380] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 00:17:47 (1790741867) [ 2564.802310] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2571.271125] loop: module loaded [ 2571.305846] loop0: detected capacity change from 0 to 4294967296 [ 2571.356428] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2573.305415] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2577.001332] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2581.168517] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2609.833578] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2609.909940] LustreError: 88199:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2609.916633] LustreError: 88199:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2609.927583] LustreError: 88199:0:(tgt_mount.c:2581:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2609.937802] LustreError: 88199:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2631.439647] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 2645.607281] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 2688.005042] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2688.275739] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2688.297590] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2688.366520] Lustre: lustre-MDT0000: new disk, initializing [ 2688.427965] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2688.433216] Lustre: Skipped 1 previous similar message [ 2688.454291] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2693.408729] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2704.358449] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2704.506779] Lustre: 91066:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2704.761360] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2709.460771] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2719.340023] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2726.200160] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2734.997451] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2735.334370] Lustre: 93036:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2741.989594] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2744.415105] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2744.595226] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2751.910156] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2757.124656] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2757.392842] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2761.543954] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2761.698848] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2765.281043] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2765.281922] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2765.283862] LustreError: Skipped 2 previous similar messages [ 2765.285796] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2765.304474] Lustre: Skipped 2 previous similar messages [ 2765.319357] Lustre: Skipped 7 previous similar messages [ 2770.114974] Lustre: server umount lustre-OST0000 complete [ 2789.343250] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2789.593809] Lustre: server umount lustre-MDT0000 complete [ 2790.888355] LustreError: 92290:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2793.903545] LustreError: 91059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790742099 with bad export cookie 15922219114311495026 [ 2793.913429] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2793.923165] LustreError: 91059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2794.425937] Lustre: server umount lustre-MDT0001 complete [ 2804.149219] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 00:21:49 (1790742109) [ 2811.038621] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2816.855131] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2825.664106] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2826.077645] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2826.092888] Lustre: Skipped 3 previous similar messages [ 2830.320853] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2837.913640] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2842.574582] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2850.339673] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2859.195175] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2869.451474] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2874.738674] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2874.968944] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2881.905971] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2882.265234] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2889.967354] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2890.296262] Lustre: lustre-OST0001: new disk, initializing [ 2890.302768] Lustre: Skipped 2 previous similar messages [ 2890.304986] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2890.307980] Lustre: Skipped 3 previous similar messages [ 2890.317995] Lustre: 98819:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2896.995830] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2897.008336] Lustre: Skipped 2 previous similar messages [ 2897.021865] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2897.149466] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2897.629625] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2906.897882] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2912.497396] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2912.722770] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2918.290675] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2918.481389] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2938.415227] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2960.644030] Lustre: Failing over lustre-MDT0000 [ 2961.164810] Lustre: server umount lustre-MDT0000 complete [ 2961.670270] LustreError: 100725:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.47@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2961.695847] LustreError: 100725:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [ 2963.941753] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2963.968670] Lustre: Skipped 5 previous similar messages [ 2979.295147] Lustre: 66813:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790742269/real 1790742269] req@000000005f80fa04 x1877728348619136/t0(0) o400->MGC192.168.202.147@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1790742285 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2979.326263] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2985.657773] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2985.663498] LDISKFS-fs (dm-0): recovery complete [ 2985.669193] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2989.566411] LustreError: 66811:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000716d43ae x1877728348626944/t0(0) o250->MGC192.168.202.147@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2989.945866] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2993.365254] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2993.367355] Lustre: 96499:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2994.177466] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2995.180707] Lustre: 95448:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2995.193387] Lustre: 95448:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 1 previous similar message [ 2995.220390] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2995.333965] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2995.663070] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2995.989525] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2996.004778] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 2996.327139] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2996.330242] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 2996.335985] Lustre: 101669:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2996.347234] Lustre: 101669:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 7 previous similar messages [ 2996.992198] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2996.999941] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 2 previous similar messages [ 2997.655227] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2997.667727] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 3 previous similar messages [ 2998.631335] Lustre: 101669:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2998.657819] Lustre: 101669:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 6 previous similar messages [ 2999.301496] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2999.322684] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 6 previous similar messages [ 2999.951149] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2999.965048] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 6 previous similar messages [ 3002.826397] Lustre: 101669:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3002.846189] Lustre: 101669:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 12 previous similar messages [ 3003.496384] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3003.515863] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 3004.151641] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3004.165270] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 3010.840635] Lustre: 101669:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3010.855545] Lustre: 101669:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 24 previous similar messages [ 3011.503503] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3011.527022] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 24 previous similar messages [ 3012.176721] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3012.186134] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 24 previous similar messages [ 3026.899198] Lustre: 101669:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3026.908837] Lustre: 101669:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 49 previous similar messages [ 3027.545578] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3027.570439] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 49 previous similar messages [ 3028.207502] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3028.221373] LustreError: 101669:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 49 previous similar messages [ 3053.207226] Lustre: 101669:0:(ldlm_lib.c:2428:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3053.222183] Lustre: 101669:0:(ldlm_lib.c:2438:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3053.229369] Lustre: 101669:0:(genops.c:1600:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 61071b44-62e9-4b9b-98de-c7368005d711@192.168.202.47@tcp [ 3053.240763] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3053.246831] Lustre: 101669:0:(ldlm_lib.c:1948:abort_req_replay_queue()) @@@ aborted: req@0000000040cd86ae x1877728333731712/t0(8589934777) o101->61071b44-62e9-4b9b-98de-c7368005d711@192.168.202.47@tcp:149/0 lens 592/0 e 0 to 0 dl 1790742369 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3053.268330] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3053.286583] Lustre: lustre-MDT0000: Denying connection for new client 61071b44-62e9-4b9b-98de-c7368005d711 (at 192.168.202.47@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3053.312419] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3053.344693] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3053.352983] Lustre: Skipped 2 previous similar messages [ 3053.355247] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3053.401208] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 3053.405722] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 3066.336714] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3066.344712] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3066.356097] Lustre: Skipped 6 previous similar messages [ 3072.278059] Lustre: server umount lustre-OST0000 complete [ 3083.119746] Lustre: server umount lustre-OST0001 complete [ 3092.476435] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3092.509166] Lustre: Skipped 5 previous similar messages [ 3102.175895] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3102.412588] Lustre: server umount lustre-MDT0000 complete [ 3102.691285] LustreError: 96176:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3102.709544] LustreError: 96176:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 41 previous similar messages [ 3106.606361] LustreError: 95427:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790742412 with bad export cookie 15922219114311546245 [ 3106.623109] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3106.626067] LustreError: 95427:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3107.420582] Lustre: server umount lustre-MDT0001 complete [ 3117.275133] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 00:27:02 (1790742422) [ 3126.756481] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3126.760762] Lustre: *** cfs_fail_loc=197, val=0*** [ 3142.239660] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3147.359541] LustreError: 103647:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.147@tcp: failed processing log, type 4: rc = -110 [ 3152.603845] Lustre: *** cfs_fail_loc=197, val=0*** [ 3152.609599] Lustre: Skipped 65 previous similar messages [ 3173.279339] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3173.283333] Lustre: Skipped 3 previous similar messages [ 3180.647397] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3188.881253] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3192.199649] Lustre: server umount lustre-OST0000 complete [ 3201.930942] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 00:28:27 (1790742507) [ 3224.014523] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3229.674141] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3239.351779] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3243.112740] LustreError: 105302:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3243.128974] LustreError: 105302:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3243.136501] LustreError: 105302:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 3243.465895] Lustre: server umount lustre-MDT0000 complete [ 3280.627739] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 3291.737651] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 3333.469716] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3333.707087] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3333.749636] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3333.893471] Lustre: lustre-MDT0000: new disk, initializing [ 3333.996484] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3339.647918] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3352.165284] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3352.344188] Lustre: 109479:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3352.639922] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3357.283658] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3367.314120] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3373.443846] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3381.938483] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3382.332623] Lustre: 111447:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3389.210369] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3389.714584] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3399.787467] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3405.881962] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3406.094672] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3413.229937] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3413.470714] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3420.131954] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3420.138520] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3420.147210] LustreError: Skipped 2 previous similar messages [ 3420.148431] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3420.148437] Lustre: Skipped 12 previous similar messages [ 3420.220144] Lustre: Skipped 1 previous similar message [ 3422.870812] Lustre: server umount lustre-OST0000 complete [ 3442.143210] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3442.392927] Lustre: server umount lustre-MDT0000 complete [ 3445.742571] LustreError: 109486:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3445.776907] LustreError: 109486:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 3447.730486] LustreError: 109469:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790742753 with bad export cookie 15922219114311556899 [ 3447.732063] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3447.744626] LustreError: 109469:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3448.264389] Lustre: server umount lustre-MDT0001 complete [ 3459.301228] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 00:32:43 (1790742763) [ 3471.330934] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 3475.161431] Key type lgssc unregistered [ 3475.607299] LNet: 113936:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3475.617281] LNetError: 113936:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3475.635957] LNet: Removed LNI 192.168.202.147@tcp [ 3476.946664] Key type .llcrypt unregistered [ 3476.948358] Key type ._llcrypt unregistered [ 3497.529739] Key type ._llcrypt registered [ 3497.532813] Key type .llcrypt registered [ 3497.717515] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 3513.889672] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 3515.194832] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3515.366098] alg: No test for adler32 (adler32-zlib) [ 3516.603656] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [ 3516.844515] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 3518.559238] Key type lgssc registered [ 3520.144866] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3564.294085] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 3564.322684] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3565.650716] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3565.702901] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3565.786389] Lustre: lustre-MDT0000: new disk, initializing [ 3565.876853] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3565.899310] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3570.309417] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3582.266409] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3582.728605] Lustre: lustre-OST0000: new disk, initializing [ 3582.735337] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3582.740493] Lustre: Skipped 1 previous similar message [ 3582.745136] Lustre: 118128:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3582.867868] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3583.157885] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3583.181619] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3583.221219] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3591.118337] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3603.962411] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3604.175386] Lustre: lustre-OST0001: new disk, initializing [ 3604.178424] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3604.182740] Lustre: 118997:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3604.268968] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3611.333839] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3611.349154] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3611.391655] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3612.793900] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3626.978076] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3627.034335] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3627.042966] Lustre: Skipped 1 previous similar message [ 3629.359355] Lustre: server umount lustre-MDT0000 complete [ 3636.010227] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3641.329032] LustreError: 117313:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790742947 with bad export cookie 8792167287117377016 [ 3641.338261] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3641.351495] LustreError: 117313:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3641.439865] Lustre: server umount lustre-OST0000 complete [ 3645.239908] Lustre: server umount lustre-OST0001 complete [ 3653.165886] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 00:35:58 (1790742958) [ 3677.229517] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 00:36:22 (1790742982) [ 3687.212819] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 3709.146895] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 00:36:54 (1790743014) [ 3745.346961] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 3754.504225] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 3789.052337] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3789.354830] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3789.401219] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3789.517985] Lustre: lustre-MDT0000: new disk, initializing [ 3789.596220] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3789.615572] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3794.762872] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3805.855046] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3806.026963] Lustre: 126745:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3806.093750] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3806.099195] Lustre: Skipped 1 previous similar message [ 3806.190319] Lustre: lustre-MDT0001: new disk, initializing [ 3806.369870] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3806.443810] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3806.472108] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3812.118643] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3820.901924] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3826.217257] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3832.456084] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3832.745367] Lustre: lustre-OST0000: new disk, initializing [ 3832.750260] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3832.754355] Lustre: 128715:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3832.894149] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3839.117341] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3839.128294] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3839.252550] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3839.393688] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3849.025813] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3854.930125] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3855.145956] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3859.962715] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3860.278566] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3865.068579] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3865.069274] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3865.090373] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3865.107134] Lustre: Skipped 2 previous similar messages [ 3865.125553] Lustre: Skipped 1 previous similar message [ 3870.181466] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3870.191708] Lustre: Skipped 1 previous similar message [ 3875.308611] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3875.313618] Lustre: Skipped 1 previous similar message [ 3876.319721] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3876.494928] Lustre: server umount lustre-OST0000 complete [ 3882.985883] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3882.992599] LustreError: Skipped 1 previous similar message [ 3882.995847] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3883.007455] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3885.540897] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3886.790852] Lustre: server umount lustre-MDT0000 complete [ 3890.471040] LustreError: 130260:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790743196 with bad export cookie 8792167287117378241 [ 3890.489095] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3890.810925] Lustre: server umount lustre-MDT0001 complete [ 3899.309691] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 00:40:04 (1790743204) [ 3905.827371] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3906.358827] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3910.531798] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3911.649871] LustreError: 131132:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3911.680359] LustreError: 131132:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 3916.768935] LustreError: 131133:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3919.382430] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3924.507965] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3932.972281] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3939.971734] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3946.731591] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3947.049537] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3947.055352] Lustre: Skipped 1 previous similar message [ 3953.295476] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3960.851364] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3968.531566] Lustre: *** cfs_fail_loc=159, val=0*** [ 3968.541148] Lustre: Skipped 1 previous similar message [ 3985.173382] Lustre: lustre-MDT0000: Client cf7a5fae-f5ae-4a39-b4aa-4b2c48fc9437 (at 192.168.202.47@tcp) reconnecting [ 3986.826293] Lustre: *** cfs_fail_loc=159, val=0*** [ 3986.832774] Lustre: Skipped 4 previous similar messages [ 4003.109667] Lustre: lustre-MDT0000: Client cf7a5fae-f5ae-4a39-b4aa-4b2c48fc9437 (at 192.168.202.47@tcp) reconnecting [ 4021.731248] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4021.731797] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4021.742331] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4021.757228] Lustre: Skipped 3 previous similar messages [ 4025.549706] Lustre: server umount lustre-OST0000 complete [ 4031.971925] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4031.986933] Lustre: Skipped 1 previous similar message [ 4036.937768] Lustre: server umount lustre-MDT0000 complete [ 4037.088987] LustreError: 133852:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4037.099245] LustreError: 133852:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4041.187085] LustreError: 131129:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4041.201583] LustreError: 131129:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4041.315383] LustreError: 131113:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790743347 with bad export cookie 8792167287117379123 [ 4041.318185] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4041.329107] LustreError: 131113:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 4041.840062] Lustre: server umount lustre-MDT0001 complete [ 4056.044821] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 4059.194587] Key type lgssc unregistered [ 4059.547325] LNet: 134908:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4059.555758] LNetError: 134908:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4059.571697] LNet: Removed LNI 192.168.202.147@tcp [ 4060.477864] Key type .llcrypt unregistered [ 4060.479583] Key type ._llcrypt unregistered [ 4084.143581] Key type ._llcrypt registered [ 4084.148583] Key type .llcrypt registered [ 4084.303740] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 4085.442223] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4085.631388] alg: No test for adler32 (adler32-zlib) [ 4086.744174] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [ 4086.983333] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 4088.639188] Key type lgssc registered [ 4089.757171] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4097.701216] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4097.760792] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4099.467061] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4105.310729] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4109.793117] LustreError: 136342:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4109.823026] LustreError: 136342:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4114.434882] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4114.770531] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4120.451633] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4130.495719] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4137.272995] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4144.023648] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4144.316873] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4145.343280] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4150.164578] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4158.787143] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4168.649518] Lustre: *** cfs_fail_loc=159, val=0*** [ 4184.810112] Lustre: lustre-MDT0000: Client 18d9a464-5323-470e-8ecb-cccc5103c1fa (at 192.168.202.47@tcp) reconnecting [ 4187.134177] Lustre: *** cfs_fail_loc=159, val=0*** [ 4202.771469] Lustre: lustre-MDT0001: Client 18d9a464-5323-470e-8ecb-cccc5103c1fa (at 192.168.202.47@tcp) reconnecting [ 4204.594048] Lustre: *** cfs_fail_loc=159, val=0*** [ 4204.600681] Lustre: Skipped 3 previous similar messages [ 4221.223244] Lustre: lustre-MDT0001: Client 18d9a464-5323-470e-8ecb-cccc5103c1fa (at 192.168.202.47@tcp) reconnecting [ 4225.941681] Lustre: 139384:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4228.532426] Lustre: *** cfs_fail_loc=159, val=0*** [ 4228.534261] Lustre: Skipped 4 previous similar messages [ 4244.723809] Lustre: lustre-MDT0000: Client 4d514649-1521-4165-91aa-ee871e21b7de (at 192.168.202.47@tcp) reconnecting [ 4246.803919] Lustre: *** cfs_fail_loc=159, val=0*** [ 4246.809909] Lustre: Skipped 13 previous similar messages [ 4263.206182] Lustre: lustre-MDT0000: Client 4d514649-1521-4165-91aa-ee871e21b7de (at 192.168.202.47@tcp) reconnecting [ 4265.333703] Lustre: 139674:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4283.874773] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4283.890428] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4283.908733] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4287.972741] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4287.980619] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4287.987198] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4287.998959] Lustre: Skipped 1 previous similar message [ 4289.248379] Lustre: server umount lustre-OST0000 complete [ 4293.090740] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4293.097451] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4293.103073] Lustre: Skipped 1 previous similar message [ 4293.128955] Lustre: Skipped 1 previous similar message [ 4298.209846] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4298.215805] Lustre: Skipped 1 previous similar message [ 4299.024256] Lustre: server umount lustre-MDT0000 complete [ 4302.914624] LustreError: 137076:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790743608 with bad export cookie 3729406800244599867 [ 4302.926493] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4302.930523] LustreError: 137076:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4303.257462] Lustre: server umount lustre-MDT0001 complete [ 4312.561571] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 4314.660537] Key type lgssc unregistered [ 4314.937825] LNet: 140753:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4314.947809] LNetError: 140753:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4314.977363] LNet: Removed LNI 192.168.202.147@tcp [ 4315.710644] Key type .llcrypt unregistered [ 4315.712581] Key type ._llcrypt unregistered [ 4335.880416] Key type ._llcrypt registered [ 4335.882174] Key type .llcrypt registered [ 4335.966679] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 4336.889185] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4336.989301] alg: No test for adler32 (adler32-zlib) [ 4337.988849] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [ 4338.246617] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 4339.903438] Key type lgssc registered [ 4341.035912] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4347.195777] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4347.207433] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4348.604144] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4353.119607] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4358.132940] LustreError: 142182:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4358.168726] LustreError: 142182:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4361.404514] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4361.916358] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4365.987555] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4373.839104] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4379.611765] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4386.392589] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4386.787736] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4391.928600] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4392.560448] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4393.979779] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4398.871498] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4405.223615] Lustre: 144760:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4407.969839] Lustre: 144808:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4409.478357] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4410.298198] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4413.634779] Lustre: 144955:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4414.383604] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4415.236927] Lustre: 145053:0:(mdt_handler.c:65:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4419.564784] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4419.576964] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4419.592763] Lustre: Skipped 1 previous similar message [ 4419.609305] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4424.675835] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4424.681020] Lustre: Skipped 1 previous similar message [ 4429.801106] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4429.807022] Lustre: Skipped 1 previous similar message [ 4433.375259] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4433.712807] Lustre: server umount lustre-OST0000 complete [ 4438.497565] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4438.515118] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4438.531499] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4440.037483] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4443.775485] Lustre: server umount lustre-MDT0000 complete [ 4445.153511] LustreError: 143238:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4445.170080] LustreError: 143238:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4447.667827] LustreError: 144857:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790743753 with bad export cookie 17568776943948919221 [ 4447.668960] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4447.677376] LustreError: 144857:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4448.020748] Lustre: server umount lustre-MDT0001 complete [ 4459.941882] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 4462.670569] Key type lgssc unregistered [ 4462.969060] LNet: 146111:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4462.973577] LNetError: 146111:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4462.995643] LNet: Removed LNI 192.168.202.147@tcp [ 4463.946473] Key type .llcrypt unregistered [ 4463.948584] Key type ._llcrypt unregistered [ 4487.078850] Key type ._llcrypt registered [ 4487.085417] Key type .llcrypt registered [ 4487.203183] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 4488.719906] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4488.792725] alg: No test for adler32 (adler32-zlib) [ 4489.935406] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [ 4490.267797] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 4491.969071] Key type lgssc registered [ 4493.577539] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4503.337166] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4503.361605] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4504.961270] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4509.427398] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4514.284411] LustreError: 147534:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4514.307412] LustreError: 147534:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4518.688452] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4519.214060] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4524.517648] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4532.941938] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4539.100925] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4546.366644] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4546.702164] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4549.826246] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4553.341082] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4560.773737] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4564.149315] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4570.339263] Lustre: *** cfs_fail_loc=159, val=0*** [ 4586.704321] Lustre: lustre-MDT0000: Client 1e8c393d-5168-4674-9840-b9289558fc70 (at 192.168.202.47@tcp) reconnecting [ 4595.190093] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4595.202519] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4595.225770] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4595.271909] Lustre: Skipped 1 previous similar message [ 4599.575550] Lustre: server umount lustre-OST0000 complete [ 4606.461949] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4606.489505] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4606.518173] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4606.526065] Lustre: Skipped 1 previous similar message [ 4610.528182] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4610.531351] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4613.375955] Lustre: server umount lustre-MDT0000 complete [ 4615.650689] LustreError: 147535:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4615.666722] LustreError: 147535:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4616.679594] LustreError: 148265:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4616.699237] LustreError: 148265:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4618.808070] LustreError: 147514:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790743924 with bad export cookie 5165948294620740782 [ 4618.813323] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4618.826182] LustreError: 147514:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4619.439139] Lustre: server umount lustre-MDT0001 complete [ 4632.288862] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 4637.057500] Key type lgssc unregistered [ 4637.470749] LNet: 151199:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4637.479686] LNetError: 151199:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4637.504493] LNet: Removed LNI 192.168.202.147@tcp [ 4638.507083] Key type .llcrypt unregistered [ 4638.510206] Key type ._llcrypt unregistered [ 4663.414250] Key type ._llcrypt registered [ 4663.421098] Key type .llcrypt registered [ 4663.578924] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 4665.457184] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4665.524249] alg: No test for adler32 (adler32-zlib) [ 4666.660716] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [ 4666.861782] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 4668.567298] Key type lgssc registered [ 4670.346037] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4679.050437] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4679.093200] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4680.550333] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4685.546059] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4690.914478] LustreError: 152631:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4690.928290] LustreError: 152631:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4695.438401] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4696.109624] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4701.587896] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4710.005261] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4718.104274] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4727.899475] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4728.611370] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4738.024331] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4740.941787] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4747.410559] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4750.204694] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4772.676694] Lustre: lustre-OST0000: evicting b856b4ad-7153-410d-9be7-c30e67bd8136 (at 192.168.202.47@tcp) by administrative request [ 4772.685864] Lustre: Skipped 1 previous similar message [ 4806.627576] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4806.628352] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4806.653138] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4806.670318] Lustre: Skipped 1 previous similar message [ 4811.749015] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4811.764061] Lustre: Skipped 1 previous similar message [ 4816.870950] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4816.888286] Lustre: Skipped 1 previous similar message [ 4817.375588] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4817.748583] Lustre: server umount lustre-OST0000 complete [ 4824.033088] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4824.036316] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4824.047361] Lustre: Skipped 1 previous similar message [ 4824.063618] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4827.115452] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4830.401496] Lustre: server umount lustre-MDT0000 complete [ 4832.227641] LustreError: 153579:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4832.239739] LustreError: 153579:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4835.206973] LustreError: 152611:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790744141 with bad export cookie 17538996503787091066 [ 4835.209679] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4835.221029] LustreError: 152611:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4835.656434] Lustre: server umount lustre-MDT0001 complete [ 4848.613667] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 4851.898749] Key type lgssc unregistered [ 4852.200845] LNet: 156706:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4852.209659] LNetError: 156706:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4853.289857] LNet: Removed LNI 192.168.202.147@tcp [ 4854.484065] Key type .llcrypt unregistered [ 4854.485676] Key type ._llcrypt unregistered [ 4938.182370] Key type ._llcrypt registered [ 4938.184497] Key type .llcrypt registered [ 4938.284288] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 4939.495652] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4939.686274] alg: No test for adler32 (adler32-zlib) [ 4940.821701] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [ 4941.025065] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 4942.634460] Key type lgssc registered [ 4943.705355] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4952.687892] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 4952.734594] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4954.413499] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4960.178910] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4964.834262] LustreError: 159212:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4964.875257] LustreError: 159212:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4969.960209] LustreError: 159213:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4971.382824] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4971.709471] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4977.238718] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4985.555568] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4992.176043] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5000.257653] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5000.938579] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5008.409849] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5016.307081] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5017.455853] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5024.512993] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5040.108458] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5040.111762] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5040.121423] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5040.158983] Lustre: Skipped 1 previous similar message [ 5040.755621] Lustre: server umount lustre-OST0000 complete [ 5048.800626] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5048.813732] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5048.836371] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5048.842913] Lustre: Skipped 1 previous similar message [ 5050.336991] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5050.343409] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5050.383431] Lustre: Skipped 1 previous similar message [ 5053.103334] Lustre: server umount lustre-MDT0000 complete [ 5055.457208] LustreError: 159213:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5055.474254] LustreError: 159213:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 5057.757039] LustreError: 159193:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790744363 with bad export cookie 5275007256194079191 [ 5057.760543] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5057.783859] LustreError: 159193:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5058.466142] Lustre: server umount lustre-MDT0001 complete [ 5071.702596] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 5075.845749] Key type lgssc unregistered [ 5076.348245] LNet: 162780:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5076.360100] LNetError: 162780:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5076.397082] LNet: Removed LNI 192.168.202.147@tcp [ 5077.757783] Key type .llcrypt unregistered [ 5077.762343] Key type ._llcrypt unregistered [ 5106.742860] Key type ._llcrypt registered [ 5106.745429] Key type .llcrypt registered [ 5106.903185] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 5107.902566] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5107.968403] alg: No test for adler32 (adler32-zlib) [ 5109.334353] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [ 5109.634679] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 5111.327321] Key type lgssc registered [ 5113.111261] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5130.640558] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 01:00:35 (1790744435) [ 5151.546962] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 5165.886748] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 5205.030729] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 5205.125626] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5206.469738] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5206.515055] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5206.651585] Lustre: lustre-MDT0000: new disk, initializing [ 5206.780996] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5206.813713] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5212.455898] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5223.601604] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5223.762272] Lustre: 167092:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5223.836599] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5223.841463] Lustre: Skipped 1 previous similar message [ 5223.908286] Lustre: lustre-MDT0001: new disk, initializing [ 5224.002472] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5224.056461] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5224.078232] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5229.553314] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5239.703297] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5248.291290] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5256.509789] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5256.859046] Lustre: lustre-OST0000: new disk, initializing [ 5256.862803] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5256.872829] Lustre: 169059:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5257.029674] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5264.524166] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5264.546774] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5264.701563] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5264.738045] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5276.033925] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5290.476762] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5290.492163] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5290.513632] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5290.526341] LustreError: Skipped 1 previous similar message [ 5292.988589] Lustre: server umount lustre-OST0000 complete [ 5300.704905] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5300.712667] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5300.731583] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5300.738379] Lustre: Skipped 2 previous similar messages [ 5300.759201] Lustre: Skipped 1 previous similar message [ 5303.581634] Lustre: server umount lustre-MDT0000 complete [ 5308.535657] LustreError: 167082:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790744614 with bad export cookie 11423122527046015641 [ 5308.547818] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5308.564325] LustreError: 167082:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5309.391292] Lustre: server umount lustre-MDT0001 complete [ 5322.273932] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 5324.963777] Key type lgssc unregistered [ 5325.277701] LNet: 170967:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5325.288129] LNetError: 170967:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5325.299824] LNet: Removed LNI 192.168.202.147@tcp [ 5326.357745] Key type .llcrypt unregistered [ 5326.360042] Key type ._llcrypt unregistered [ 5350.307460] Key type ._llcrypt registered [ 5350.309312] Key type .llcrypt registered [ 5350.488819] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 5351.714394] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5352.047812] alg: No test for adler32 (adler32-zlib) [ 5353.378067] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [ 5353.663822] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 5355.384228] Key type lgssc registered [ 5357.472219] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5367.299446] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 5367.311814] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5368.722454] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5374.861355] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5379.042592] LustreError: 172388:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5379.076584] LustreError: 172388:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 5384.161582] LustreError: 172389:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5386.656583] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5387.353881] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5393.450559] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5402.297654] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5409.709929] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5417.921862] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5418.341911] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5426.224576] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5434.624438] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5438.918556] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5445.868361] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5445.901908] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5445.924072] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5445.947806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.964245] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5447.095803] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5447.116242] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5447.137436] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5447.142558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.162215] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5448.294626] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5448.307930] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5448.321576] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5448.328376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.350303] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5449.249186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.057511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.809936] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5450.816691] Lustre: Skipped 2 previous similar messages [ 5450.834416] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5450.850453] Lustre: Skipped 2 previous similar messages [ 5450.865162] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5450.875860] Lustre: Skipped 2 previous similar messages [ 5450.880159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.894767] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5450.897366] Lustre: Skipped 2 previous similar messages [ 5451.881547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.815339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.817865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.525860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.518075] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5455.522564] Lustre: Skipped 4 previous similar messages [ 5455.549972] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5455.559170] Lustre: Skipped 4 previous similar messages [ 5455.564122] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5455.567255] Lustre: Skipped 4 previous similar messages [ 5455.571167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.582071] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5455.596847] Lustre: Skipped 4 previous similar messages [ 5456.424982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.473380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.742900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.861925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.181143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.101152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.234259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.531214] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5464.547760] Lustre: Skipped 7 previous similar messages [ 5464.578752] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5464.588642] Lustre: Skipped 7 previous similar messages [ 5464.599525] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5464.613951] Lustre: Skipped 7 previous similar messages [ 5464.624747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.637649] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5464.640310] Lustre: Skipped 7 previous similar messages [ 5465.472506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.522450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.712157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.702099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.610565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.655744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.944378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.883457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.134705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.689912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.915753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.301309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.002030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.788066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.865689] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5480.873349] Lustre: Skipped 14 previous similar messages [ 5480.905403] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5480.916568] Lustre: Skipped 14 previous similar messages [ 5480.919952] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5480.926100] Lustre: Skipped 14 previous similar messages [ 5480.932366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.940901] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5480.943554] Lustre: Skipped 14 previous similar messages [ 5482.118463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.554643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.541477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.036981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.160902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.240324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.507385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.381688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.170046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.028725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.886716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.803229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.983825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.137975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.090331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.232795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.280766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.188303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.239549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.438850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.445418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.495523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.484715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.704499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.849190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.727768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.814659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.609213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.624342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.559990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.562969] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5513.573438] Lustre: Skipped 30 previous similar messages [ 5513.593551] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5513.602889] Lustre: Skipped 30 previous similar messages [ 5513.606987] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5513.611421] Lustre: Skipped 30 previous similar messages [ 5513.614938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.628449] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5513.634550] Lustre: Skipped 30 previous similar messages [ 5514.531945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.555468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.462711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.486096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.626621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.814686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.972993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.069681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.817092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.707416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.509392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.624438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.469395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.242040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.956459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.981586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.033676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.941746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.934646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.899263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.074636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.037475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.053030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.095639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.068495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.502220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.659808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.580257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.898147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.185461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.496599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.619557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.661504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.755543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.238338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.487401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.562983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.493396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.796464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.965481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.934222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.163839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.294266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.192729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.236543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.185092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.249075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.394965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.496738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.535614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.544825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.643759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.706628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.770218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.888987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.860988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.847281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.787225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.758806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.590069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.719280] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5577.725267] Lustre: Skipped 60 previous similar messages [ 5577.734744] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5577.746763] Lustre: Skipped 60 previous similar messages [ 5577.750132] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5577.753359] Lustre: Skipped 60 previous similar messages [ 5577.757809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.771631] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5577.784924] Lustre: Skipped 60 previous similar messages [ 5578.663101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.576740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.362754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.511285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.636500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.750598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.784181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.686956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.877604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.825543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.210531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.149343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.150130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.031610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.015461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.126426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.443620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.357545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.597883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.769145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.869205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.441392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.712064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.803487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.605108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.515579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.956518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.087908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.297086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.644587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.808433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.700208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.630056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.570226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.550821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.565642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.639100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.718407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.716133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.948558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.789865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.987729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.185958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.399366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.745985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.623624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.688641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.769126] LustreError: 172408:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5629.792445] LustreError: 172408:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5629.793794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.787739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.604556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.637989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.928415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.830529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.285920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.576716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.864210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.108367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.366937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.355606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.175803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.178180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.225529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.258273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.578931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.751160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.538047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.427381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.000146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.119429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.177649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.111456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.874959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.736630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.639638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.799497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.931589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.942684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.775592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.515991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.541656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.601371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.626641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.907291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.848217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.176334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.541869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.595523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.816656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.578434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.606901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.508840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.404430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.590305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.802790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.123515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.052151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.165202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.376519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.747385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.072578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.327487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.686423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.115363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.511389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.748348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.750764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.707461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.768904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.929460] LustreError: 172408:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5695.955979] LustreError: 172408:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5695.958665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.216943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.401684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.629827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.125172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.331345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.675891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.932302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.689294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.919303] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5706.929301] Lustre: Skipped 117 previous similar messages [ 5706.947761] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5706.960423] Lustre: Skipped 117 previous similar messages [ 5706.965087] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5706.969466] Lustre: Skipped 117 previous similar messages [ 5706.972949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.980344] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5706.987287] Lustre: Skipped 117 previous similar messages [ 5707.943673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.931788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.925117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.982951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.155571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.910035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.885096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.715478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.874983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.944595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.129704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.429119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.480787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.696820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.870269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.946847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.190297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.178885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.101976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.056547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.806533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.760683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.998740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.895061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.912697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.932391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.032492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.931517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.816712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.063552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.225751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.257183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.489411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.493259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.504235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.222306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.019839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.837142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.844493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.762149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.762104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.809459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.743398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.635553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.615146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.503932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.333762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.301656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.284120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.162117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.063163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.029082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.843329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.740150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.815081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.601640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.290714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.035968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.646886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.594612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.469610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.204832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.066490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.937945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.774451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.784192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.695022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.641153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.572802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.625101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.587590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.703416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.573343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.554700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.453490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.413237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.221819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.969124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.790381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.498052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.324046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.137770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.862579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.838100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.642576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.310393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.927645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.559877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.245955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.931107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.727542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.415772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.098877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.812902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.539844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.177990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.868242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.617988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.377468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.279374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.022592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.877656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.736621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.664921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.636986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.438253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.298916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.252797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.085885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.952530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.760616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.513832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.369363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.246553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.063276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.852778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.772915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.603989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.516734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.430905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.396724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.291837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.325418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.392748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.295413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.343076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.514195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.774963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.765639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.662564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.503556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.547258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.535093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.498756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.558192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.642796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.771667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.708481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.672756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.721668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.576356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.507113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.557364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.540293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.823729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.703861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.756581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.616232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.711282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.667740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.645276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.661701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.782723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.633523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.456443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.403928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.352544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.086201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.925129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.651303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.596386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.265187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.190645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.033868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.940363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.799099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.713573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.504924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.507759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.430177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.392452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.378236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.393709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.572337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.522962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.643557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.517379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.433293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.388979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.412748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.575848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.637378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.570635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.527796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.411679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.285485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.310095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.305396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.323619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.301742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.290509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.486978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.203523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.037681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.899797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.734805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.661244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.791267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.606656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.512178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.348484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.397305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.363484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.333563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.201420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.975256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.826087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.769868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.588315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.502705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.211656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.094320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.940570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.861435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.734584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.624815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.367363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.280583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.231652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.139037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.978358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.869270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.774892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.835814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.891947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.913647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.104994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.205303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.306954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.235113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.169989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.291500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.419376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.348075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.455246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.406341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.359347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.204786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.047259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.780911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.638859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.418207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.272797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.294201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.276769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.251730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.337473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.289907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.423140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.360267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.673620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.629694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.540531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.646901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.679795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.733758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.959791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.996963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.994233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.029287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.799690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.854711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.880961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.917782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.813227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.936786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.973062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.018972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.032687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.067657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.105100] LustreError: 172408:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5961.116647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.118234] LustreError: 172408:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5962.126045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.263464] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5963.269878] Lustre: Skipped 272 previous similar messages [ 5963.297923] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5963.308449] Lustre: Skipped 272 previous similar messages [ 5963.313171] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5963.317970] Lustre: Skipped 272 previous similar messages [ 5963.320757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.338380] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5963.376482] Lustre: Skipped 272 previous similar messages [ 5964.240295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.278699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.148805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.172434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.209864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.259582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.324460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.353824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.471975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.571557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.594619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.567941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.442657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.486231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.263151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.234940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.296811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.397335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.346440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.295394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.545060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.646864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.852101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.826415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.979832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.030602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.031934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.179588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.072416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.180560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.154464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.219892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.174253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.194880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.184835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.195487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.075117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.987059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.774243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.756674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.521552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.478084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.440432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.499094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.419371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.446236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.305104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.336873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.266250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.043658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.040764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.118283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.091342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.257787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.423561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.388691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.293633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.158682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.285699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.316441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.420593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.466970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.549780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.661453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.684452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.751549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.750778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.756804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.768769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.926522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.970788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.178815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.217503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.164192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.136348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.105376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.092859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.978910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.001709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.845917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.983107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.934923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.811715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.756738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.626559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.419197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.522616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.433679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.441505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.438553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.153600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.925560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.870634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.734043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.720598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.483275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.443200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.366636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.395265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.311512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.246533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.195449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.248570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.103527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.067446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.979802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.998570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.881585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.793595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.687608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.725273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.644906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.615175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.480139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.501255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.495477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.612420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.797905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.721169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.011554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.917152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.919357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.806086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.731585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.938039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.080691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.077778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.085948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.034888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.088256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.054336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.116269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.127193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.966971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.843526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.673267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.615821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.586640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.488946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.625262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.701984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.886570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.011654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.125349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.252475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.234348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.117268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.216967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.326657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.119404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.278207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.422763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.623283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.758649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.850088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.732232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.596332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.427527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.342805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.163231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.268418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.293933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.028232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.002725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.907875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.780985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.655264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.676734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.583351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.742832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.656800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.763627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.820235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.752302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.952890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.943872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.008319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.920568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.905217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.683439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.792483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.777186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.702284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.658414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.522175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.313975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.214179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.173106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.081498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.763180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.641128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.447765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.266768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.043532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.989326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.905525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.006303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.884360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.760909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.494752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.346389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.299351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.344092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.408106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.450986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.276963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.135250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.973981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.066981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.943204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.936289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.800275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.918941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.966358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.295873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.171499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.304830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.260846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.311350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.274905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.193522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.062439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.899444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.934217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.877621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.798149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.651840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.550211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.551150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.592950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.626946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.751708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.666204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.830530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.004490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.910605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.869281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.773994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.773173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.682264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.764039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.602241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.470654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.220672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.103903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.026581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.035740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.846841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.866939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.579689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.464766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.317792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.252000] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.076829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.909786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.760454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.739301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.639567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.530816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.497842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.413494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.453354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.305687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.301098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.348223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.305630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.220564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.009195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.865506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.752969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.587414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.628929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.420336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.395159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.174399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.968448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.783691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.710853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.452967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.518840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.614994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.433515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.427102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.497875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.479635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.433355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.400586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.294993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.122953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.076494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.112274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.144599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.184672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.041960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.006306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.200502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.180222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.370900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.583481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.681640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.575221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.826807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.234448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.288342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.504399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.530676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.711288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.086836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.305440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.468122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.706771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.800859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.396770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.741373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.054708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.409634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.490315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.415499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.417685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.375292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.484583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.369975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.515847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.323900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.414742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.526289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.563669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.688491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.961237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.113655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.185558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.276353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.349511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.575027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.519313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.671936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.637346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.514755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.891346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.964770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.151345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.108073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.099906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.158221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.182397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.195681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.345587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.407630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.507811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.672506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.811830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.859929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.791411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.779267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.728168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.690891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.925530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.069913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.070954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.097962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.082882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.102128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.073080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.311614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.270704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.503728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.486471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.486957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.538418] LustreError: 172408:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6330.550660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.552447] LustreError: 172408:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6331.658607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.680447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.633857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.582946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.194851] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6516.208025] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6516.215208] Lustre: Skipped 374 previous similar messages [ 6516.237130] Lustre: Skipped 1 previous similar message [ 6521.314089] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6522.061397] Lustre: server umount lustre-OST0000 complete [ 6528.992933] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6529.000434] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6529.007970] Lustre: Skipped 1 previous similar message [ 6531.557312] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6531.569334] Lustre: Skipped 1 previous similar message [ 6536.679578] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6541.279190] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6541.523300] Lustre: server umount lustre-MDT0000 complete [ 6541.792506] LustreError: 172389:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6541.812187] LustreError: 172389:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 6545.762426] LustreError: 172368:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790745851 with bad export cookie 3141515502978181131 [ 6545.766850] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6545.772649] LustreError: 172368:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 6546.208230] Lustre: server umount lustre-MDT0001 complete [ 6559.701243] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 6562.660847] Key type lgssc unregistered [ 6563.149436] LNet: 221866:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6563.161690] LNetError: 221866:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6563.181477] LNet: Removed LNI 192.168.202.147@tcp [ 6564.217747] Key type .llcrypt unregistered [ 6564.220109] Key type ._llcrypt unregistered [ 6591.589963] Key type ._llcrypt registered [ 6591.595983] Key type .llcrypt registered [ 6591.706955] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 6593.275832] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6593.332693] alg: No test for adler32 (adler32-zlib) [ 6594.596684] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [ 6595.026667] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 6596.711335] Key type lgssc registered [ 6598.236745] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6607.012459] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 6607.035678] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6608.699113] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6613.788975] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 6618.081119] LustreError: 223297:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6618.103728] LustreError: 223297:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 6623.204843] LustreError: 223298:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6626.032945] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6626.352346] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6631.304912] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 6640.376133] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6648.787167] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6659.024897] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6659.518094] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6666.562949] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 6675.936084] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6679.037530] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:17075 to 0x280000401:17121) [ 6685.968502] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6688.320489] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6688.340720] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6688.352774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6688.373928] LustreError: 223317:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6688.375727] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6692.321164] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6692.341772] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6692.393579] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6694.385557] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6694.396419] Lustre: Skipped 1 previous similar message [ 6694.411579] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6696.885816] Lustre: server umount lustre-OST0000 complete [ 6708.391358] LustreError: 223317:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6733.060834] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6733.646925] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6734.646740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6734.659705] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6742.200799] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 6744.941360] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6744.957091] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 6749.809267] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6760.250577] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 8 sec [ 6766.623575] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9df886580000.ost_server_uuid 50 [ 6768.033636] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9df886580000.ost_server_uuid in FULL state after 0 sec [ 6773.216242] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6773.240533] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6773.265443] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6773.273729] Lustre: Skipped 1 previous similar message [ 6774.762857] Lustre: server umount lustre-OST0000 complete [ 6779.878846] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6779.884612] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6779.917847] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6786.032606] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6786.043488] Lustre: Skipped 3 previous similar messages [ 6791.137252] Lustre: 222599:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790746081/real 1790746081] req@00000000e5163a20 x1877733172899072/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1790746097 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 6791.138164] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6791.199950] Lustre: 222599:0:(client.c:2504:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 6791.231738] Lustre: Skipped 1 previous similar message [ 6793.183374] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6793.487671] Lustre: server umount lustre-MDT0000 complete [ 6796.277234] LustreError: 223293:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6796.321533] LustreError: 223293:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 6796.397060] Lustre: 222600:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790746086/real 1790746086] req@0000000000a76d95 x1877733172900864/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1790746102 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 6798.745508] LustreError: 223277:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790746104 with bad export cookie 2024299602230001279 [ 6798.757459] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6798.766397] LustreError: 223277:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6799.279511] Lustre: server umount lustre-MDT0001 complete [ 6813.965789] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 6818.000118] Key type lgssc unregistered [ 6818.395630] LNet: 228509:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6818.404259] LNetError: 228509:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6818.417917] LNet: Removed LNI 192.168.202.147@tcp [ 6819.309360] Key type .llcrypt unregistered [ 6819.312718] Key type ._llcrypt unregistered [ 6854.088696] Key type ._llcrypt registered [ 6854.091208] Key type .llcrypt registered [ 6854.205698] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 6855.603793] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6855.668451] alg: No test for adler32 (adler32-zlib) [ 6856.817341] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [ 6857.163283] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 6858.911259] Key type lgssc registered [ 6860.659785] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6872.699414] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 6872.711734] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6874.158735] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6879.579496] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 6884.323516] LustreError: 230257:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6884.344638] LustreError: 230257:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 6888.552659] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6888.979617] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6893.528421] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 6896.969954] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6905.387649] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6905.940802] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6913.620042] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 6919.365608] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:17123 to 0x280000401:17153) [ 6923.304841] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6923.619850] Lustre: lustre-OST0001: new disk, initializing [ 6923.638266] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6923.653430] Lustre: 232561:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6923.788705] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6929.579616] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6929.599395] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6929.704153] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6931.930494] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 6945.526779] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6954.158542] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6970.850558] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6970.869834] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6970.871855] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6970.894732] Lustre: Skipped 3 previous similar messages [ 6975.974161] LustreError: 232747:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6975.986966] LustreError: 232747:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 6976.234105] Lustre: server umount lustre-MDT0000 complete [ 6980.840013] LustreError: 232573:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790746286 with bad export cookie 1506726236917805661 [ 6980.862365] LustreError: 232573:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6980.875217] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6981.096458] LustreError: 231761:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6981.101089] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6981.110118] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6981.110132] Lustre: Skipped 3 previous similar messages [ 6981.129721] LustreError: 231761:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 6981.193603] Lustre: Skipped 1 previous similar message [ 6981.425151] Lustre: server umount lustre-MDT0001 complete [ 6988.879122] Lustre: server umount lustre-OST0000 complete [ 6993.545172] Lustre: server umount lustre-OST0001 complete [ 7021.019170] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 7032.954058] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7033.262992] LustreError: 235671:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7033.342100] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7038.321682] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7038.433649] LustreError: 235672:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7048.284097] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7048.592107] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7053.105225] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7056.167248] Lustre: 236813:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7063.132126] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7070.086774] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7073.714240] LustreError: 237166:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7073.766740] LustreError: 237166:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 7078.849640] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:17156 to 0x280000401:17185) [ 7081.240790] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7081.468305] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7081.473734] Lustre: Skipped 1 previous similar message [ 7091.428382] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7099.405811] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7103.609807] Lustre: 238685:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7115.233275] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7115.252784] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7115.283191] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7115.290785] Lustre: Skipped 1 previous similar message [ 7117.285584] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7117.285613] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7117.295932] Lustre: Skipped 1 previous similar message [ 7117.309303] Lustre: Skipped 3 previous similar messages [ 7121.203497] Lustre: server umount mylustre-MDT0000 complete [ 7122.410367] LustreError: 237188:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7122.462678] LustreError: 237188:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 7125.226626] LustreError: 237996:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790746431 with bad export cookie 1506726236917808468 [ 7125.234310] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7125.245882] LustreError: 237996:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7125.750038] Lustre: server umount mylustre-MDT0001 complete [ 7139.262795] Lustre: server umount mylustre-OST0000 complete [ 7153.642461] Lustre: server umount mylustre-OST0001 complete [ 7179.146470] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 7191.427419] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7192.067775] LustreError: 240996:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7192.183425] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7197.554969] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7207.539188] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7212.774196] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7216.032394] Lustre: 242139:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7223.307391] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7231.584305] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7237.186158] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:17156 to 0x280000401:17217) [ 7241.532044] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7246.851335] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 7250.270661] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7258.688658] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7262.625707] Lustre: 244011:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7272.420858] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7272.430102] LustreError: 240997:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7272.441758] Lustre: Skipped 3 previous similar messages [ 7272.482248] LustreError: 240997:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 8 previous similar messages [ 7272.534482] Lustre: server umount tfs-MDT0000 complete [ 7277.211189] LustreError: 240971:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790746583 with bad export cookie 1506726236917810134 [ 7277.224463] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7277.742351] Lustre: server umount tfs-MDT0001 complete [ 7284.777213] Lustre: server umount tfs-OST0000 complete [ 7289.789538] Lustre: server umount tfs-OST0001 complete [ 7313.103520] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 7323.860548] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7324.368789] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7324.374744] Lustre: Skipped 3 previous similar messages [ 7329.239585] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7338.941918] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7344.390877] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7347.583610] Lustre: 247422:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7355.108621] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7363.081558] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7369.866192] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:17156 to 0x280000401:17249) [ 7372.278815] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7377.924150] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 7379.392713] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7388.552213] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7392.954956] Lustre: 249291:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7401.470829] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 01:38:26 (1790746706) [ 7418.853586] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7418.855209] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7418.867731] Lustre: Skipped 4 previous similar messages [ 7418.874881] Lustre: Skipped 2 previous similar messages [ 7425.007828] Lustre: server umount lustre-MDT0000 complete [ 7429.099908] LustreError: 247283:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7429.129288] LustreError: 247283:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 12 previous similar messages [ 7429.614266] LustreError: 246255:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790746735 with bad export cookie 1506726236917811800 [ 7429.617714] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7429.627783] LustreError: 246255:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 7434.217408] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7434.231954] Lustre: Skipped 5 previous similar messages [ 7435.930760] Lustre: server umount lustre-MDT0001 complete [ 7446.123697] Lustre: server umount lustre-OST0000 complete [ 7456.607620] Lustre: server umount lustre-OST0001 complete [ 7464.767306] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 7477.323328] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 7522.174497] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7522.668757] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7522.737438] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7522.827974] Lustre: lustre-MDT0000: new disk, initializing [ 7522.886637] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7522.893556] Lustre: Skipped 3 previous similar messages [ 7522.909392] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7527.898616] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7540.094253] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7540.158386] Lustre: 252477:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7540.198261] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7540.203081] Lustre: Skipped 1 previous similar message [ 7540.251877] Lustre: lustre-MDT0001: new disk, initializing [ 7540.340074] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7540.347491] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7546.070655] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7558.616982] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7558.963515] Lustre: lustre-OST0000: new disk, initializing [ 7558.968629] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7558.977977] Lustre: 254061:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7560.117780] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7560.131243] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7560.276881] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7566.179484] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7577.954039] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7578.117303] Lustre: lustre-OST0001: new disk, initializing [ 7578.120643] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7578.127339] Lustre: 254932:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7580.138242] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7580.154991] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7580.331752] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7585.388954] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7601.296467] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 01:41:45 (1790746905) [ 7618.590105] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 01:42:02 (1790746922) [ 7636.467077] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7636.482703] Lustre: Skipped 4 previous similar messages [ 7636.491511] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7636.494537] Lustre: Skipped 1 previous similar message [ 7639.406344] Lustre: server umount lustre-OST0000 complete [ 7649.822471] Lustre: server umount lustre-MDT0000 complete [ 7653.740481] LustreError: 252470:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790746959 with bad export cookie 1506726236917813354 [ 7653.751383] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7653.770599] LustreError: 252470:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7668.194110] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7668.514861] Lustre: server umount lustre-MDT0001 complete [ 7681.460995] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 7756.083417] Lustre: server umount lustre-OST0001 complete [ 7762.813872] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 7779.330718] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 7824.567993] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7824.883394] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7824.956239] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7825.081040] Lustre: lustre-MDT0000: new disk, initializing [ 7825.192748] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7825.199787] Lustre: Skipped 3 previous similar messages [ 7825.216636] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7830.709177] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7842.521687] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7842.655654] Lustre: 259788:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7842.758137] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7842.770526] Lustre: Skipped 1 previous similar message [ 7842.915554] Lustre: lustre-MDT0001: new disk, initializing [ 7843.141528] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7843.154689] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7848.651168] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7860.306933] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7867.401183] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7876.446565] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7876.899204] Lustre: lustre-OST0000: new disk, initializing [ 7876.906824] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7876.914548] Lustre: 261759:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7878.420115] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7878.436332] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7878.647682] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7884.940714] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7896.597816] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7909.347400] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7909.348695] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7909.364269] Lustre: Skipped 5 previous similar messages [ 7909.373024] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7909.373033] Lustre: Skipped 7 previous similar messages [ 7911.182167] Lustre: server umount lustre-OST0000 complete [ 7922.250353] Lustre: server umount lustre-MDT0000 complete [ 7924.705064] LustreError: 259801:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7924.721794] LustreError: 259801:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 14 previous similar messages [ 7926.423263] LustreError: 259781:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790747232 with bad export cookie 1506726236917814824 [ 7926.430508] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7926.436459] LustreError: 259781:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7926.704804] Lustre: server umount lustre-MDT0001 complete [ 7941.298324] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 7945.086366] Key type lgssc unregistered [ 7945.777081] LNet: 263519:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7945.796712] LNetError: 263519:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7945.826944] LNet: Removed LNI 192.168.202.147@tcp [ 7947.585302] Key type .llcrypt unregistered [ 7947.587290] Key type ._llcrypt unregistered [ 7982.661526] Key type ._llcrypt registered [ 7982.663790] Key type .llcrypt registered [ 7982.820487] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 7984.453972] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7984.495237] alg: No test for adler32 (adler32-zlib) [ 7986.156864] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [ 7986.393670] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 7988.048697] Key type lgssc registered [ 7989.850405] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7999.957529] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 7999.986958] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8001.454190] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8006.744447] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8011.748654] LustreError: 265001:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8011.757527] LustreError: 265001:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 8015.498678] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8015.748528] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8021.756553] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8033.655641] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8042.762814] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8055.246221] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8055.997756] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8065.925798] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8071.491523] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8077.284711] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8078.339129] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8080.865345] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8080.872885] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8080.892561] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8081.890679] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8081.892498] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8081.895095] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8086.461588] Lustre: server umount lustre-OST0000 complete [ 8092.129583] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8092.133097] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8092.140814] Lustre: Skipped 1 previous similar message [ 8092.144058] Lustre: Skipped 1 previous similar message [ 8097.248707] LustreError: 265001:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8097.294587] Lustre: server umount lustre-MDT0000 complete [ 8102.370020] LustreError: 265001:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8102.401540] LustreError: 265001:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 8102.630883] LustreError: 264982:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790747408 with bad export cookie 2194320210588032521 [ 8102.660752] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8103.328978] Lustre: server umount lustre-MDT0001 complete [ 8117.955292] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 8122.248700] Key type lgssc unregistered [ 8122.663584] LNet: 268590:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8122.678646] LNetError: 268590:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8122.703410] LNet: Removed LNI 192.168.202.147@tcp [ 8123.976613] Key type .llcrypt unregistered [ 8123.986934] Key type ._llcrypt unregistered [ 8144.175092] Key type ._llcrypt registered [ 8144.188328] Key type .llcrypt registered [ 8144.342223] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 8145.988700] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8146.118059] alg: No test for adler32 (adler32-zlib) [ 8147.257521] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [ 8147.578965] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 8149.297194] Key type lgssc registered [ 8150.999981] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8161.966499] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 8161.988744] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8177.569662] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8182.687331] LustreError: 269799:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.147@tcp: failed processing log, type 4: rc = -110 [ 8209.376052] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8216.478846] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8223.102225] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8226.393697] Lustre: server umount lustre-OST0000 complete [ 8247.927784] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 8251.674137] Key type lgssc unregistered [ 8252.120645] LNet: 271429:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8252.152576] LNetError: 271429:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8253.222721] LNet: Removed LNI 192.168.202.147@tcp [ 8254.373458] Key type .llcrypt unregistered [ 8254.376495] Key type ._llcrypt unregistered [ 8299.041653] Key type ._llcrypt registered [ 8299.043047] Key type .llcrypt registered [ 8299.149595] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 8300.536188] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8300.685463] alg: No test for adler32 (adler32-zlib) [ 8301.770259] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [ 8301.994994] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 8303.687207] Key type lgssc registered [ 8304.971462] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8320.075923] loop0: detected capacity change from 0 to 204800000 [ 8320.305360] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8320.315427] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8320.671509] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8322.076385] loop0: detected capacity change from 0 to 204800000 [ 8322.356762] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8322.378211] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8322.741438] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8323.952198] loop0: detected capacity change from 0 to 204800000 [ 8324.167458] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8324.186621] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8324.263132] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8325.766498] loop0: detected capacity change from 0 to 204800000 [ 8325.962692] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8325.974324] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8326.262044] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8327.678586] loop0: detected capacity change from 0 to 204800000 [ 8327.707733] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8331.878680] loop0: detected capacity change from 0 to 204800000 [ 8331.913335] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8335.990772] loop0: detected capacity change from 0 to 204800000 [ 8336.045086] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8340.372918] loop0: detected capacity change from 0 to 204800000 [ 8340.410510] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8344.919399] loop0: detected capacity change from 0 to 204800000 [ 8345.078499] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 8345.108864] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8345.160156] Lustre: 274865:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8345.178544] Lustre: 274865:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8345.188134] Lustre: 274865:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8345.202949] Lustre: 274865:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8345.209521] Lustre: 274865:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8345.217772] Lustre: 274865:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8345.225233] Lustre: 274865:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8345.231136] Lustre: 274865:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8345.237790] Lustre: 274865:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8345.243836] Lustre: 274865:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8345.254141] Lustre: 274865:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8345.263189] Lustre: 274865:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8345.270213] Lustre: 274865:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8345.277629] Lustre: 274865:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8352.705182] Lustre: server umount MGS complete [ 8354.096951] loop0: detected capacity change from 0 to 204800000 [ 8354.220844] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8355.091738] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8355.147743] Lustre: lustre-MDT0000: Aborting client recovery [ 8356.349281] loop1: detected capacity change from 0 to 204800000 [ 8356.489204] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8356.576928] Lustre: 275300:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8356.586105] Lustre: 275300:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8356.592822] Lustre: 275300:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8356.600892] Lustre: 275300:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8356.618299] Lustre: 275300:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8356.641528] Lustre: 275300:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8356.675156] Lustre: 275300:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8356.732032] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8357.050881] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8357.103733] Lustre: lustre-MDT0001: Aborting client recovery [ 8358.398734] loop2: detected capacity change from 0 to 204800000 [ 8358.536426] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8358.586854] Lustre: 275380:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8358.604986] Lustre: 275380:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8358.617178] Lustre: 275380:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8358.630619] Lustre: 275380:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8358.642609] Lustre: 275380:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8358.657091] Lustre: 275380:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8358.673944] Lustre: 275380:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8358.789564] Lustre: Found index 0 for lustre-OST0000, updating log [ 8358.842265] Lustre: 275399:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8358.899181] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8358.929859] Lustre: lustre-OST0000: Aborting recovery [ 8360.010682] loop3: detected capacity change from 0 to 204800000 [ 8360.232179] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8360.299200] Lustre: 275464:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8360.315591] Lustre: 275464:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8360.331546] Lustre: 275464:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8360.346331] Lustre: 275464:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8360.355445] Lustre: 275464:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8360.362566] Lustre: 275464:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8360.373896] Lustre: 275464:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8360.410093] Lustre: Found index 1 for lustre-OST0001, updating log [ 8360.482240] Lustre: 275469:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8360.602507] Lustre: lustre-OST0001: Aborting recovery [ 8364.718566] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8364.730340] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8364.744566] LustreError: 275244:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 8364.816402] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8364.821387] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8366.756877] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8366.760357] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8368.674419] LustreError: 275211:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 490us: rc = -2 [ 8369.177694] LustreError: 275209:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1911us: rc = -2 [ 8369.189935] LustreError: 275209:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 19 previous similar messages [ 8382.435675] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8382.442328] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8382.445079] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8382.479373] Lustre: Skipped 3 previous similar messages [ 8387.553756] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8387.559459] Lustre: Skipped 6 previous similar messages [ 8392.675419] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8392.688717] Lustre: Skipped 3 previous similar messages [ 8396.255916] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8397.792802] LustreError: 275210:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8397.831458] LustreError: 275210:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 8398.590927] Lustre: server umount lustre-MDT0000 complete [ 8399.766821] LustreError: 275194:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790747705 with bad export cookie 15935693072411454922 [ 8399.798967] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8402.913528] LustreError: 275317:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8402.915501] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8402.943366] LustreError: 275317:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 8402.955721] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8402.965347] Lustre: Skipped 1 previous similar message [ 8404.968950] LustreError: 275317:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8404.994985] LustreError: 275317:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 8410.079734] LustreError: 275208:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8410.079815] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8410.103449] LustreError: 275208:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 8410.109468] Lustre: Skipped 3 previous similar messages [ 8414.175136] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8414.600574] Lustre: server umount lustre-MDT0001 complete [ 8421.888203] Lustre: server umount lustre-OST0000 complete [ 8437.215139] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8437.436677] Lustre: server umount lustre-OST0001 complete [ 8446.154300] Lustre: DEBUG MARKER: == conf-sanity test 108c: mkfs.lustre --mmp option enables MMP feature ========================================================== 01:55:51 (1790747751) [ 8468.460882] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 8509.484370] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8509.652949] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8509.673503] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8509.765803] Lustre: lustre-MDT0000: new disk, initializing [ 8509.846377] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8509.850755] Lustre: Skipped 1 previous similar message [ 8509.864666] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8514.463592] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8522.441873] Lustre: server umount lustre-MDT0000 complete [ 8542.582631] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 01:57:27 (1790747847) [ 8573.769905] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 8584.267197] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 8624.489961] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8624.719654] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8624.758261] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8624.765851] Lustre: Skipped 1 previous similar message [ 8624.836466] Lustre: lustre-MDT0000: new disk, initializing [ 8624.893409] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8624.907596] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8629.989343] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8642.108895] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8642.292347] Lustre: 282578:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8642.374767] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8642.378762] Lustre: Skipped 1 previous similar message [ 8642.461608] Lustre: lustre-MDT0001: new disk, initializing [ 8642.594712] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8642.673440] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8642.694325] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8648.416466] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8659.281951] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8666.953731] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8674.594717] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8674.827367] Lustre: lustre-OST0000: new disk, initializing [ 8674.830462] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8674.835654] Lustre: 284551:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8674.939777] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8680.546895] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8680.580079] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8680.867857] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8683.189339] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8692.910633] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8698.605993] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8711.872449] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8718.447850] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8762.852134] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8762.854159] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8762.857773] LustreError: Skipped 1 previous similar message [ 8762.859329] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8762.873281] Lustre: Skipped 2 previous similar messages [ 8775.135179] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8775.383792] Lustre: server umount lustre-OST0000 complete [ 8780.775604] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8780.785504] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8780.801045] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8780.808744] Lustre: Skipped 5 previous similar messages [ 8786.443284] Lustre: server umount lustre-MDT0000 complete [ 8788.454535] LustreError: 282590:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8788.477442] LustreError: 282590:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 8790.734667] LustreError: 282569:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790748096 with bad export cookie 15935693072411457876 [ 8790.740524] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8790.743907] LustreError: 282569:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 8791.109382] Lustre: server umount lustre-MDT0001 complete [ 8801.281785] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8806.106051] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8810.648035] Lustre: server umount MGS complete [ 8821.081888] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8821.533067] LustreError: 288426:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8821.570618] LustreError: 288439:0:(qmt_pool.c:1648:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8821.635154] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8826.057849] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8834.605603] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8839.608228] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8847.305569] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8854.241318] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8861.909521] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8869.205576] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8877.354970] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8902.134553] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8902.153090] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8902.173733] Lustre: Skipped 1 previous similar message [ 8902.177925] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8902.182477] Lustre: Skipped 2 previous similar messages [ 8907.158359] Lustre: server umount lustre-OST0000 complete [ 8915.424814] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8915.483635] Lustre: Skipped 1 previous similar message [ 8926.175137] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8926.558921] Lustre: server umount lustre-MDT0000 complete [ 8930.786733] LustreError: 288422:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8930.824646] LustreError: 288422:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 8932.493026] LustreError: 288406:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790748238 with bad export cookie 15935693072411459710 [ 8932.522525] LustreError: 288406:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8932.532185] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8933.288945] Lustre: server umount lustre-MDT0001 complete [ 8944.777693] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 8947.528994] Key type lgssc unregistered [ 8947.789626] LNet: 292370:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8947.797441] LNetError: 292370:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8947.819134] LNet: Removed LNI 192.168.202.147@tcp [ 8948.631702] Key type .llcrypt unregistered [ 8948.633351] Key type ._llcrypt unregistered [ 8993.199535] Key type ._llcrypt registered [ 8993.201765] Key type .llcrypt registered [ 8993.265462] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 9011.052434] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 9012.934937] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9013.115810] alg: No test for adler32 (adler32-zlib) [ 9014.189224] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [ 9014.356355] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 9016.007496] Key type lgssc registered [ 9017.712818] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9061.247400] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 9061.274581] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9062.581175] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9062.646813] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9062.772399] Lustre: lustre-MDT0000: new disk, initializing [ 9062.874691] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9062.886846] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9067.141236] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 9078.646454] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9078.912437] Lustre: 296348:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9079.021598] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9079.027850] Lustre: Skipped 1 previous similar message [ 9079.187341] Lustre: lustre-MDT0001: new disk, initializing [ 9079.312614] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9079.370964] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9079.389518] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9085.225861] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 9095.518086] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9102.562819] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9111.423056] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9111.818308] Lustre: lustre-OST0000: new disk, initializing [ 9111.821913] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9111.830529] Lustre: 298321:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9111.922242] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9118.942938] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 9119.288272] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9119.304150] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9119.451959] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9129.038669] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9136.416506] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9150.022909] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9154.075772] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9159.839441] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9206.760307] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9206.760684] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9206.766311] LustreError: Skipped 1 previous similar message [ 9206.784465] Lustre: Skipped 1 previous similar message [ 9206.803387] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9208.595070] Lustre: server umount lustre-OST0000 complete [ 9216.992512] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9217.027405] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9217.032360] Lustre: Skipped 1 previous similar message [ 9218.830591] Lustre: server umount lustre-MDT0000 complete [ 9222.651424] LustreError: 296340:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790748528 with bad export cookie 5640606995214517924 [ 9222.656252] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9222.660454] LustreError: 296340:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9222.983169] Lustre: server umount lustre-MDT0001 complete [ 9235.253888] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9241.270705] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 9246.662453] Lustre: server umount MGS complete [ 9260.727360] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9261.264369] LustreError: 302192:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9261.281970] LustreError: 302205:0:(qmt_pool.c:1648:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9261.284151] LustreError: 302192:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 9261.359135] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9266.658689] LustreError: 302193:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9268.246541] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 9271.793688] LustreError: 302192:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9276.904908] LustreError: 302193:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9279.521518] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9279.864227] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9283.998956] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 9291.641446] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9298.577630] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9306.265131] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9306.617132] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9314.092700] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 9323.177118] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9355.744221] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9355.754649] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9355.776184] Lustre: Skipped 1 previous similar message [ 9355.784783] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9359.720460] Lustre: server umount lustre-OST0000 complete [ 9365.991068] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9365.999165] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9366.013432] Lustre: Skipped 2 previous similar messages [ 9366.037930] Lustre: Skipped 2 previous similar messages [ 9371.105197] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9371.116187] Lustre: Skipped 1 previous similar message [ 9378.271182] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9378.374329] Lustre: server umount lustre-MDT0000 complete [ 9380.320724] LustreError: 302192:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9380.337581] LustreError: 302192:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9382.658570] LustreError: 302174:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790748688 with bad export cookie 5640606995214519849 [ 9382.676424] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9382.683079] LustreError: 302174:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9383.040398] Lustre: server umount lustre-MDT0001 complete [ 9398.240764] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 9402.760540] Key type lgssc unregistered [ 9403.145790] LNet: 306141:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9403.170930] LNetError: 306141:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9403.204653] LNet: Removed LNI 192.168.202.147@tcp [ 9404.196809] Key type .llcrypt unregistered [ 9404.199663] Key type ._llcrypt unregistered [ 9428.931387] Key type ._llcrypt registered [ 9428.936474] Key type .llcrypt registered [ 9429.097108] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 9445.118372] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 9446.717730] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9446.955691] alg: No test for adler32 (adler32-zlib) [ 9448.215696] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [ 9448.608907] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 9450.375534] Key type lgssc registered [ 9451.596848] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9478.753401] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 9496.723582] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [ 9496.760399] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9498.121042] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9498.244442] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9498.349627] Lustre: lustre-MDT0000: new disk, initializing [ 9498.477196] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9498.500394] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9503.620235] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 9515.879206] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9516.115827] Lustre: lustre-OST0000: new disk, initializing [ 9516.120448] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9516.123056] Lustre: Skipped 1 previous similar message [ 9516.128164] Lustre: 310209:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9516.239307] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9524.125626] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 9524.406293] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9524.426659] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9524.473882] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10476.318374] Lustre: server umount lustre-MDT0000 complete [10480.920345] LustreError: 309393:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790749786 with bad export cookie 14777327600163735363 [10480.926358] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10490.455841] Lustre: server umount lustre-OST0000 complete [10516.636608] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [10526.056565] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [10567.403321] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10567.744365] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10567.771820] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10567.856330] Lustre: lustre-MDT0000: new disk, initializing [10567.946315] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10567.961067] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10573.118211] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10585.299965] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10585.494394] Lustre: 314568:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10585.582876] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10585.588439] Lustre: Skipped 1 previous similar message [10585.680273] Lustre: lustre-MDT0001: new disk, initializing [10585.780484] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10585.843088] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10585.860185] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10591.329262] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10601.317547] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10606.978527] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10613.357207] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10613.584639] Lustre: lustre-OST0000: new disk, initializing [10613.590235] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10613.595629] Lustre: 316538:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10613.673085] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10620.617980] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10623.053410] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10623.068840] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10623.131721] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10630.552612] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10635.959239] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10636.254286] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10642.117736] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10642.314284] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10649.066108] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10649.069486] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10649.076613] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10649.086452] LustreError: Skipped 1 previous similar message [10649.128325] Lustre: Skipped 1 previous similar message [10650.661859] Lustre: server umount lustre-OST0000 complete [10657.760580] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10657.779160] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10657.790495] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10657.794877] Lustre: Skipped 1 previous similar message [10659.299840] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10659.300599] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10659.335178] Lustre: Skipped 1 previous similar message [10661.184827] Lustre: server umount lustre-MDT0000 complete [10665.719525] LustreError: 316537:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790749971 with bad export cookie 14777327600166676182 [10665.733850] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10665.737061] LustreError: 316537:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10666.477271] Lustre: server umount lustre-MDT0001 complete [10670.385415] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10672.267923] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 02:32:57 (1790749977) [10678.817497] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10679.220693] LustreError: 318851:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10679.245681] LustreError: 318851:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [10679.290433] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10683.642958] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10684.389024] LustreError: 318852:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10689.510412] LustreError: 318851:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10692.526942] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10693.097687] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10698.222060] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10707.256793] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10713.736188] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10721.535422] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10721.929962] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10728.662329] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10737.128534] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10744.885200] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10745.081853] Lustre: lustre-OST0001: new disk, initializing [10745.095941] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10745.108179] Lustre: 321686:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10745.232451] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10752.095495] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10752.114619] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10752.167662] LustreError: 321988:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10752.889869] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10753.185890] LustreError: 321988:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10753.196728] LustreError: 321988:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10754.284882] LustreError: 321988:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10754.306184] LustreError: 321988:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10756.321199] LustreError: 321995:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10756.327751] LustreError: 321995:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10760.449645] LustreError: 321995:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10760.479869] LustreError: 321995:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [10763.138441] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10763.373508] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10768.682634] LustreError: 321988:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10768.694226] LustreError: 321988:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10769.275111] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9df886721800.ost_server_uuid 50 [10770.787285] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9df886721800.ost_server_uuid in FULL state after 0 sec [10776.626971] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9df886721800.ost_server_uuid 50 [10778.316864] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9df886721800.ost_server_uuid in FULL state after 0 sec [10780.083321] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10795.489975] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10795.511255] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10795.517577] LustreError: Skipped 1 previous similar message [10795.520340] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10795.542830] Lustre: Skipped 1 previous similar message [10803.710769] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10803.716248] Lustre: Skipped 5 previous similar messages [10808.289041] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10808.518464] Lustre: server umount lustre-OST0001 complete [10808.807908] LustreError: 321989:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10808.817770] LustreError: 321989:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10813.925746] LustreError: 321989:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10813.954103] LustreError: 321989:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [10817.508012] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10817.514204] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10817.528793] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10817.536104] Lustre: Skipped 1 previous similar message [10821.782913] Lustre: server umount lustre-OST0000 complete [10826.208516] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10826.214122] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10826.246109] Lustre: Skipped 1 previous similar message [10832.314427] Lustre: server umount lustre-MDT0000 complete [10834.414830] LustreError: 319580:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10834.444018] LustreError: 319580:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [10837.945099] LustreError: 318876:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790750143 with bad export cookie 14777327600166677064 [10837.951432] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10837.957133] LustreError: 318876:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10838.477895] Lustre: server umount lustre-MDT0001 complete [10857.821419] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [10862.550735] Key type lgssc unregistered [10863.057039] LNet: 324400:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10863.067728] LNetError: 324400:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10863.093527] LNet: Removed LNI 192.168.202.147@tcp [10864.496770] Key type .llcrypt unregistered [10864.498901] Key type ._llcrypt unregistered [10890.389394] Key type ._llcrypt registered [10890.391211] Key type .llcrypt registered [10890.506481] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [10891.435691] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10891.475319] alg: No test for adler32 (adler32-zlib) [10892.672625] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [10892.973383] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [10894.679241] Key type lgssc registered [10896.172480] Lustre: Echo OBD driver; http://www.lustre.org/ [10904.800934] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [10904.819891] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10906.377941] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10911.351098] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10916.839878] LustreError: 325837:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10916.860025] LustreError: 325837:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [10920.734415] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10921.955204] LustreError: 325838:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10927.077516] LustreError: 325837:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10929.503529] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10929.851195] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10935.404830] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10946.023706] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10946.524113] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10951.607924] LustreError: 327517:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10951.641144] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10953.575763] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10959.847039] LustreError: 327785:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10959.877317] LustreError: 327785:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [10962.776543] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10970.090349] LustreError: 327517:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10970.105472] LustreError: 327517:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [10970.185661] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10970.482984] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10975.751920] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10977.831378] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10986.320117] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10994.557326] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10994.909102] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11103.712329] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11103.716787] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11103.735352] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11103.756456] LustreError: Skipped 1 previous similar message [11103.766132] Lustre: Skipped 1 previous similar message [11106.637042] Lustre: server umount lustre-OST0001 complete [11108.846726] LustreError: 327519:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11108.871666] LustreError: 327519:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11113.963264] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11113.972352] Lustre: Skipped 1 previous similar message [11113.975989] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11118.562608] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11118.570308] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11118.590213] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11124.202412] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11124.212250] Lustre: Skipped 3 previous similar messages [11128.287416] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11128.538399] Lustre: server umount lustre-OST0000 complete [11134.432649] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11134.451237] Lustre: Skipped 1 previous similar message [11134.451885] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11134.452639] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11134.481634] Lustre: Skipped 1 previous similar message [11139.277420] Lustre: server umount lustre-MDT0000 complete [11143.474025] LustreError: 326769:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790750449 with bad export cookie 12782382499430172069 [11143.482487] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11143.493345] LustreError: 326769:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11143.774962] Lustre: server umount lustre-MDT0001 complete [11155.916286] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [11159.433618] Key type lgssc unregistered [11159.876783] LNet: 330783:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11159.884039] LNetError: 330783:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11159.897874] LNet: Removed LNI 192.168.202.147@tcp [11160.992094] Key type .llcrypt unregistered [11160.996092] Key type ._llcrypt unregistered [11213.063609] Key type ._llcrypt registered [11213.068548] Key type .llcrypt registered [11213.230971] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [11215.007415] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11215.098580] alg: No test for adler32 (adler32-zlib) [11216.236281] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [11216.549682] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [11218.239226] Key type lgssc registered [11219.675737] Lustre: Echo OBD driver; http://www.lustre.org/ [11229.928884] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [11229.959715] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11231.663952] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11237.123703] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11241.956802] LustreError: 333091:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11241.972536] LustreError: 333091:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11245.939796] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11246.412138] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11251.245501] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11254.292877] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11261.101918] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11261.488635] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11268.439374] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11271.723375] LustreError: 334577:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11276.775410] LustreError: 334580:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11277.712978] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11277.831743] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11278.081215] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11283.502819] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11284.549029] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11294.753393] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11306.140978] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11323.363740] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11323.377081] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11323.396100] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11324.391418] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11324.396565] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11324.411452] Lustre: Skipped 2 previous similar messages [11324.420478] Lustre: Skipped 2 previous similar messages [11326.966978] Lustre: server umount lustre-MDT0000 complete [11329.519432] LustreError: 333087:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11329.569757] LustreError: 333087:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [11331.649510] LustreError: 336098:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790750637 with bad export cookie 14642727194513470986 [11331.669343] LustreError: 336098:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11331.672257] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11331.970398] Lustre: server umount lustre-MDT0001 complete [11346.139081] Lustre: server umount lustre-OST0000 complete [11350.111109] Lustre: 332344:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1790750640/real 1790750640] req@ffff9110d4b5ed80 x1877738019424384/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1790750656 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11350.167526] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11351.355130] Lustre: server umount lustre-OST0001 complete [11373.974686] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [11386.843058] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11387.393703] LustreError: 338354:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11387.439589] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11390.952127] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11397.609221] LustreError: 338354:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11397.639267] LustreError: 338354:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11399.504020] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11400.241992] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11406.319423] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11409.989099] Lustre: 339496:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11417.530595] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11417.939901] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11424.111698] LustreError: 339850:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11424.914841] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11432.446763] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11435.580919] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11441.162407] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11442.937656] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11451.231478] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11455.063879] Lustre: 341367:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11461.612170] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11461.629188] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11461.630217] Lustre: Skipped 3 previous similar messages [11461.633427] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11461.640641] Lustre: Skipped 4 previous similar messages [11464.771292] Lustre: server umount lustre-MDT0000 complete [11466.726162] LustreError: 338349:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11466.757516] LustreError: 338349:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [11468.492712] LustreError: 338334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790750774 with bad export cookie 14642727194513472680 [11468.499319] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11468.506804] LustreError: 338334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11468.829695] Lustre: server umount lustre-MDT0001 complete [11483.431158] Lustre: server umount lustre-OST0000 complete [11496.493684] Lustre: server umount lustre-OST0001 complete [11500.372227] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11501.999584] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11504.253261] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 02:46:49 (1790750809) [11532.955495] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [11544.533159] loop0: detected capacity change from 0 to 1099511627776 [11545.527856] SGI XFS with ACLs, security attributes, quota, no debug enabled [11545.541308] *** VALIDATE xfs *** [11545.578668] XFS (loop0): Mounting V5 Filesystem [11545.662285] XFS (loop0): Ending clean mount [11547.491424] loop1: detected capacity change from 0 to 18691697672192 [11667.627454] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [11677.569909] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [11719.358277] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11719.651470] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11719.679187] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11719.746892] Lustre: lustre-MDT0000: new disk, initializing [11719.847726] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11719.855197] Lustre: Skipped 1 previous similar message [11719.870923] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11724.684679] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11735.324640] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11735.428727] Lustre: 346507:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11735.478139] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11735.480509] Lustre: Skipped 1 previous similar message [11735.528429] Lustre: lustre-MDT0001: new disk, initializing [11735.660152] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11735.678238] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11740.581385] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11751.449415] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11758.192594] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11765.628736] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11766.072970] Lustre: lustre-OST0000: new disk, initializing [11766.078419] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11766.084867] Lustre: 348476:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11767.918078] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11767.930979] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11768.041221] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11772.629436] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11783.763563] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11789.313538] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11789.678084] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11795.238170] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11795.537928] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11799.010936] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11799.017287] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11799.022275] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11799.036821] Lustre: Skipped 1 previous similar message [11799.053515] Lustre: Skipped 1 previous similar message [11804.130194] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11804.144498] Lustre: Skipped 1 previous similar message [11812.319180] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11812.606652] Lustre: server umount lustre-OST0000 complete [11817.442943] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11817.446563] LustreError: Skipped 1 previous similar message [11817.455671] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11817.477689] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11817.480925] Lustre: Skipped 2 previous similar messages [11822.580296] Lustre: server umount lustre-MDT0000 complete [11824.611324] LustreError: 346519:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11824.629253] LustreError: 346519:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11826.756953] LustreError: 346498:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790751132 with bad export cookie 14642727194513474227 [11826.766251] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11826.768594] LustreError: 346498:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11827.104915] Lustre: server umount lustre-MDT0001 complete [11839.119234] XFS (loop0): Unmounting Filesystem [11844.832465] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 02:52:29 (1790751149) [11851.794898] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11852.591495] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11852.597561] Lustre: Skipped 2 previous similar messages [11857.881082] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11867.991146] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11873.290506] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11881.148255] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11886.495181] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11893.387310] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11900.763797] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11908.483358] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11923.937176] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11923.945086] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11923.955652] LustreError: Skipped 1 previous similar message [11923.962857] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11923.991357] Lustre: Skipped 2 previous similar messages [11924.034059] Lustre: Skipped 3 previous similar messages [11927.591891] Lustre: server umount lustre-OST0000 complete [11938.641467] Lustre: server umount lustre-MDT0000 complete [11942.595610] LustreError: 351020:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790751248 with bad export cookie 14642727194513475116 [11942.612603] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11942.614327] LustreError: 351020:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11943.169564] Lustre: server umount lustre-MDT0001 complete [11954.780994] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [11957.830457] Key type lgssc unregistered [11958.268017] LNet: 354666:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11958.292475] LNetError: 354666:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11959.334673] LNet: Removed LNI 192.168.202.147@tcp [11960.689783] Key type .llcrypt unregistered [11960.694699] Key type ._llcrypt unregistered [11988.424480] Key type ._llcrypt registered [11988.426353] Key type .llcrypt registered [11988.496969] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [11989.523104] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11989.546842] alg: No test for adler32 (adler32-zlib) [11990.602054] Lustre: Lustre: Build Version: 2.17.59_3_g94a7b9b [11990.867960] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [11992.519200] Key type lgssc registered [11993.701784] Lustre: Echo OBD driver; http://www.lustre.org/ [12001.732492] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' or install '{/usr}/sbin/mount.lustre_tgt' [12001.743336] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12003.123101] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12007.550200] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [12013.539832] LustreError: 356251:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12013.559756] LustreError: 356251:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [12016.407411] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12016.878643] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12022.500339] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [12031.410178] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12040.050508] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12048.516481] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12048.862605] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12055.999829] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [12066.172239] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12076.513686] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12076.527407] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12079.850854] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.47@tcp (stopping) [12079.856985] Lustre: Skipped 1 previous similar message [12081.641872] LustreError: 358094:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12081.792522] Lustre: server umount lustre-MDT0001 complete [12084.941493] LustreError: 356246:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.47@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12084.967275] LustreError: 356246:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [12090.069755] LustreError: 358094:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.47@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12090.093438] LustreError: 358094:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [12093.111886] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12093.189130] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12093.191201] Lustre: Found index 1 for lustre-MDT0001, updating log [12093.207589] Lustre: 356241:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12093.511678] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12098.152809] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [12098.553859] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12101.643747] LustreError: 359853:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12101.659096] LustreError: 359853:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12101.674227] LustreError: 356243:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12101.794515] LustreError: 359856:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.147@tcp: cfg command failed: rc = -17 [12101.814910] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12101.814910] [12405.731272] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12405.733505] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12405.749472] Lustre: Skipped 1 previous similar message [12405.751567] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12410.861405] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12410.870732] Lustre: Skipped 2 previous similar messages [12411.798042] Lustre: server umount lustre-MDT0001 complete [12412.639769] LustreError: 360172:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.47@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12412.668503] LustreError: 360172:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [12421.096040] LustreError: 356246:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12421.131376] LustreError: 356246:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [12424.539278] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12424.685453] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12424.692866] Lustre: Found index 1 for lustre-MDT0001, updating log [12424.740426] Lustre: 356239:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12425.150516] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12430.350320] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12430.873314] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [12431.375598] LustreError: 367958:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12431.388774] LustreError: 367958:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12431.404215] LustreError: 356243:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12431.410887] LustreError: 356243:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12431.506559] LustreError: 367966:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12431.515345] LustreError: 367966:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12431.523571] LustreError: 367966:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.147@tcp: cfg command failed: rc = -17 [12431.532583] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.147@tcp [12431.532583] [12737.508885] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12737.519759] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12737.547099] Lustre: Skipped 1 previous similar message [12737.553320] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12742.904097] Lustre: server umount lustre-MDT0001 complete [12743.648437] LustreError: 356979:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12743.671033] LustreError: 356979:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [12754.483438] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12754.574707] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12754.579797] Lustre: Found index 1 for lustre-MDT0001, updating log [12754.620843] Lustre: 356241:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12754.914389] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12758.421233] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [12760.047329] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12764.158800] LustreError: 376226:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12764.163659] LustreError: 376226:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12764.174223] LustreError: 356243:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12764.182142] LustreError: 356243:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12764.291181] LustreError: 376229:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12764.323071] LustreError: 376229:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.147@tcp: cfg command failed: rc = -114 [12764.337609] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12764.337609] [13082.603804] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13082.604851] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13082.606170] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13082.627749] Lustre: Skipped 3 previous similar messages [13082.647517] Lustre: Skipped 5 previous similar messages [13084.665409] Lustre: server umount lustre-MDT0000 complete [13087.719135] LustreError: 356247:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13087.733766] LustreError: 356247:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 8 previous similar messages [13088.025557] LustreError: 356986:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790752393 with bad export cookie 5099009601883054411 [13088.030730] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13088.032854] LustreError: 356986:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13092.835260] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13094.479786] Lustre: server umount lustre-MDT0001 complete [13098.855794] Lustre: server umount lustre-OST0000 complete [13107.976298] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [13118.861976] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [13158.707778] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 03:14:23 (1790752463) [13166.512679] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13167.049718] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13167.124107] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13167.248480] Lustre: lustre-MDT0000: new disk, initializing [13167.377085] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13167.447701] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13171.782766] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [13182.830982] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13182.978237] Lustre: 386444:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13183.043861] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13183.053456] Lustre: Skipped 1 previous similar message [13183.141824] Lustre: lustre-MDT0001: new disk, initializing [13183.232964] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13183.264185] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13183.273497] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13187.714128] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [13198.044876] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13203.998036] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13210.954323] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13211.256491] Lustre: lustre-OST0000: new disk, initializing [13211.261767] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13211.271492] Lustre: 388414:0:(osd_compat.c:1354:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13211.416284] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13217.828462] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [13218.383701] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13218.396854] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13218.545829] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13228.339789] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13239.267337] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13239.275629] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13239.282506] Lustre: Skipped 1 previous similar message [13239.301390] Lustre: Skipped 4 previous similar messages [13244.387362] LustreError: 386458:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13244.405628] LustreError: 386458:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [13244.529914] Lustre: server umount lustre-MDT0000 complete [13248.671048] LustreError: 388413:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1790752554 with bad export cookie 5099009601883056364 [13248.672122] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13248.680552] LustreError: 388413:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13249.152676] Lustre: server umount lustre-MDT0001 complete [13264.161587] Lustre: server umount lustre-OST0000 complete [13294.468436] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [13312.147161] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [13348.945827] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 13095 sec ========= 03:17:33 (1790752653) [13350.897516] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 03:17:35 (1790752655) === [13354.985310] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 03:17:40 (1790752660) === [13401.463169] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [13404.660935] Key type lgssc unregistered [13405.023694] LNet: 394801:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13405.035763] LNetError: 394801:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13405.078159] LNet: Removed LNI 192.168.202.147@tcp [13405.926347] Key type .llcrypt unregistered [13405.929769] Key type ._llcrypt unregistered