[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 480802569 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f5b30-0x000f5b3f] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F5950 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE1BB7 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE1A53 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 001A13 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE1AC7 000090 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE1B57 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE1B8F 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe1a53-0xbffe1ac6] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe1a52] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe1ac7-0xbffe1b56] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe1b57-0xbffe1b8e] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe1b8f-0xbffe1bb6] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.002383] x2apic enabled [ 0.003009] Switched APIC routing to physical x2apic. [ 0.004015] kvm-guest: setup PV IPIs [ 0.006926] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007022] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008016] pid_max: default: 32768 minimum: 301 [ 0.009159] LSM: Security Framework initializing [ 0.010060] Yama: becoming mindful. [ 0.011050] SELinux: Initializing. [ 0.012060] *** VALIDATE selinux *** [ 0.020792] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.024727] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.025137] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027098] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029106] *** VALIDATE tmpfs *** [ 0.031384] *** VALIDATE proc *** [ 0.032325] *** VALIDATE cgroup *** [ 0.033011] *** VALIDATE cgroup2 *** [ 0.034296] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035277] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036020] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037042] Spectre V2 : User space: Vulnerable [ 0.038013] Speculative Store Bypass: Vulnerable [ 0.041672] debug: unmapping init [mem 0xffffffff88e59000-0xffffffff88e60fff] [ 0.043243] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044874] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045028] ... version: 2 [ 0.046022] ... bit width: 48 [ 0.047017] ... generic registers: 4 [ 0.048022] ... value mask: 0000ffffffffffff [ 0.049025] ... max period: 00007fffffffffff [ 0.050022] ... fixed-purpose events: 3 [ 0.051021] ... event mask: 000000070000000f [ 0.052450] rcu: Hierarchical SRCU implementation. [ 0.054530] smp: Bringing up secondary CPUs ... [ 0.055703] x86: Booting SMP configuration: [ 0.056041] .... node #0, CPUs: #1 #2 #3 [ 0.059423] smp: Brought up 1 node, 4 CPUs [ 0.061013] smpboot: Max logical packages: 1 [ 0.062018] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.145627] node 0 deferred pages initialised in 80ms [ 0.149185] devtmpfs: initialized [ 0.150245] x86/mm: Memory block size: 128MB [ 0.152711] gcov: version magic: 0x41383552 [ 0.154278] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.155080] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.156347] pinctrl core: initialized pinctrl subsystem [ 0.157219] [ 0.157629] ************************************************************* [ 0.158012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.159024] ** ** [ 0.160018] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.161015] ** ** [ 0.162019] ** This means that this kernel is built to expose internal ** [ 0.163014] ** IOMMU data structures, which may compromise security on ** [ 0.164015] ** your system. ** [ 0.165011] ** ** [ 0.166016] ** If you see this message and you are not debugging the ** [ 0.167018] ** kernel, report this immediately to your vendor! ** [ 0.168014] ** ** [ 0.169013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.170014] ************************************************************* [ 0.171654] NET: Registered protocol family 16 [ 0.172435] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.173074] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.174068] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.175514] cpuidle: using governor menu [ 0.177946] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.179436] PCI: Using configuration type 1 for base access [ 0.181153] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.191059] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.192023] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.194040] cryptd: max_cpu_qlen set to 1000 [ 0.196307] ACPI: Added _OSI(Module Device) [ 0.197015] ACPI: Added _OSI(Processor Device) [ 0.198022] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.199018] ACPI: Added _OSI(Processor Aggregator Device) [ 0.202435] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.206081] ACPI: Interpreter enabled [ 0.207086] ACPI: PM: (supports S0 S3 S4 S5) [ 0.208023] ACPI: Using IOAPIC for interrupt routing [ 0.209185] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.210449] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.219838] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.220051] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.221038] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.222084] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.224410] acpiphp: Slot [2] registered [ 0.225149] acpiphp: Slot [3] registered [ 0.226103] acpiphp: Slot [4] registered [ 0.227127] acpiphp: Slot [5] registered [ 0.228134] acpiphp: Slot [6] registered [ 0.229128] acpiphp: Slot [7] registered [ 0.230168] acpiphp: Slot [8] registered [ 0.231119] acpiphp: Slot [9] registered [ 0.232117] acpiphp: Slot [10] registered [ 0.233110] acpiphp: Slot [11] registered [ 0.234180] acpiphp: Slot [12] registered [ 0.235180] acpiphp: Slot [13] registered [ 0.236207] acpiphp: Slot [14] registered [ 0.237138] acpiphp: Slot [15] registered [ 0.238131] acpiphp: Slot [16] registered [ 0.239108] acpiphp: Slot [17] registered [ 0.240133] acpiphp: Slot [18] registered [ 0.241119] acpiphp: Slot [19] registered [ 0.242158] acpiphp: Slot [20] registered [ 0.243115] acpiphp: Slot [21] registered [ 0.244100] acpiphp: Slot [22] registered [ 0.245110] acpiphp: Slot [23] registered [ 0.246119] acpiphp: Slot [24] registered [ 0.247179] acpiphp: Slot [25] registered [ 0.248137] acpiphp: Slot [26] registered [ 0.249222] acpiphp: Slot [27] registered [ 0.250112] acpiphp: Slot [28] registered [ 0.251095] acpiphp: Slot [29] registered [ 0.252119] acpiphp: Slot [30] registered [ 0.253123] acpiphp: Slot [31] registered [ 0.254081] PCI host bridge to bus 0000:00 [ 0.255025] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.256027] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.257025] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.258032] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.259020] pci_bus 0000:00: root bus resource [mem 0x180000000-0x1ffffffff window] [ 0.260034] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.262185] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.265352] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.268748] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.274017] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.277055] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.278022] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.279018] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.280019] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.281490] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.282831] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.283046] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.284683] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.286966] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.293945] pci 0000:00:02.0: reg 0x20: [mem 0xfebe4000-0xfebe7fff 64bit pref] [ 0.295883] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.298483] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.301018] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.304021] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.312020] pci 0000:00:05.0: reg 0x20: [mem 0xfebe8000-0xfebebfff 64bit pref] [ 0.317835] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.320025] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.323018] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.330019] pci 0000:00:06.0: reg 0x20: [mem 0xfebec000-0xfebeffff 64bit pref] [ 0.335000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.348029] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.356027] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.374020] pci 0000:00:07.0: reg 0x20: [mem 0xfebf0000-0xfebf3fff 64bit pref] [ 0.384664] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.392021] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.399023] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.410017] pci 0000:00:08.0: reg 0x20: [mem 0xfebf4000-0xfebf7fff 64bit pref] [ 0.420383] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.428016] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.437023] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.452019] pci 0000:00:09.0: reg 0x20: [mem 0xfebf8000-0xfebfbfff 64bit pref] [ 0.464851] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.472018] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.478025] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.491023] pci 0000:00:0a.0: reg 0x20: [mem 0xfebfc000-0xfebfffff 64bit pref] [ 0.503640] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.508639] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.511366] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.513377] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.515275] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.522208] iommu: Default domain type: Passthrough [ 0.524448] SCSI subsystem initialized [ 0.526204] ACPI: bus type USB registered [ 0.528138] usbcore: registered new interface driver usbfs [ 0.530099] usbcore: registered new interface driver hub [ 0.531103] usbcore: registered new device driver usb [ 0.533267] pps_core: LinuxPPS API ver. 1 registered [ 0.535018] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.537048] PTP clock support registered [ 0.538078] EDAC MC: Ver: 3.0.0 [ 0.541120] PCI: Using ACPI for IRQ routing [ 0.542943] NetLabel: Initializing [ 0.543009] NetLabel: domain hash size = 128 [ 0.544011] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.545095] NetLabel: unlabeled traffic allowed by default [ 0.547165] vgaarb: loaded [ 0.548172] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.551012] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.561000] clocksource: Switched to clocksource kvm-clock [ 0.667160] VFS: Disk quotas dquot_6.6.0 [ 0.668306] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.670551] *** VALIDATE ramfs *** [ 0.671883] *** VALIDATE hugetlbfs *** [ 0.673337] pnp: PnP ACPI init [ 0.675612] pnp: PnP ACPI: found 6 devices [ 0.697542] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.700626] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.702480] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.704864] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.707432] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.710219] pci_bus 0000:00: resource 8 [mem 0x180000000-0x1ffffffff window] [ 0.713090] NET: Registered protocol family 2 [ 0.715646] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.720422] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.724322] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.729632] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.732649] TCP: Hash tables configured (established 65536 bind 65536) [ 0.735634] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.738827] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.740878] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.744132] NET: Registered protocol family 1 [ 0.747564] RPC: Registered named UNIX socket transport module. [ 0.750066] RPC: Registered udp transport module. [ 0.752099] RPC: Registered tcp transport module. [ 0.754045] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.756617] NET: Registered protocol family 44 [ 0.758284] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.760584] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.762960] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.765310] PCI: CLS 0 bytes, default 64 [ 0.767177] Unpacking initramfs... [ 2.213954] debug: unmapping init [mem 0xffff88c47cc54000-0xffff88c47ffbffff] [ 2.219608] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.221842] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.225506] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.750769] Initialise system trusted keyrings [ 2.752482] Key type blacklist registered [ 2.755345] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.765578] zbud: loaded [ 2.771304] *** VALIDATE nfs *** [ 2.772639] *** VALIDATE nfs4 *** [ 2.774749] pstore: using deflate compression [ 2.778202] Platform Keyring initialized [ 2.883875] NET: Registered protocol family 38 [ 2.885729] Key type asymmetric registered [ 2.887103] Asymmetric key parser 'x509' registered [ 2.888858] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.891851] io scheduler mq-deadline registered [ 2.894020] io scheduler kyber registered [ 2.895925] io scheduler bfq registered [ 2.897734] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.900907] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.903070] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.905675] ACPI: Power Button [PWRF] [ 2.997282] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.088466] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.273427] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.377590] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.571747] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.605387] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.643023] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.648950] Non-volatile memory driver v1.3 [ 3.650666] Linux agpgart interface v0.103 [ 3.683327] virtio_blk virtio1: [vda] 134032 512-byte logical blocks (68.6 MB/65.4 MiB) [ 3.687616] vda: detected capacity change from 0 to 68624384 [ 3.702536] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.705653] vdb: detected capacity change from 0 to 1073741824 [ 3.721470] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.725022] vdc: detected capacity change from 0 to 2621440000 [ 3.739238] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.742414] vdd: detected capacity change from 0 to 2621440000 [ 3.756146] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.761182] vde: detected capacity change from 0 to 4294967296 [ 3.775314] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.778086] vdf: detected capacity change from 0 to 4294967296 [ 3.786722] libphy: Fixed MDIO Bus: probed [ 3.794550] usbcore: registered new interface driver usbserial_generic [ 3.797268] usbserial: USB Serial support registered for generic [ 3.799903] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.804548] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.806067] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.808511] mousedev: PS/2 mouse device common for all mice [ 3.810995] rtc_cmos 00:05: RTC can wake from S4 [ 3.814137] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.818418] rtc_cmos 00:05: registered as rtc0 [ 3.819916] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.823994] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.833743] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.833906] intel_pstate: CPU model not supported [ 3.839847] hid: raw HID events driver (C) Jiri Kosina [ 3.842216] usbcore: registered new interface driver usbhid [ 3.844434] usbhid: USB HID core driver [ 3.846217] drop_monitor: Initializing network drop monitor service [ 3.848577] Initializing XFRM netlink socket [ 3.850545] NET: Registered protocol family 10 [ 3.853424] Segment Routing with IPv6 [ 3.854887] NET: Registered protocol family 17 [ 3.857117] mpls_gso: MPLS GSO support [ 3.863362] RAS: Correctable Errors collector initialized. [ 3.865768] AVX version of gcm_enc/dec engaged. [ 3.870931] AES CTR mode by8 optimization enabled [ 3.964248] sched_clock: Marking stable (3964210845, 0)->(5083253833, -1119042988) [ 3.967994] registered taskstats version 1 [ 3.969908] Loading compiled-in X.509 certificates [ 3.971873] zswap: loaded using pool lzo/zbud [ 4.006760] Key type big_key registered [ 4.020035] Key type encrypted registered [ 4.022211] ima: No TPM chip found, activating TPM-bypass! [ 4.024268] ima: Allocated hash algorithm: sha1 [ 4.026181] ima: No architecture policies found [ 4.028118] evm: Initialising EVM extended attributes: [ 4.029841] evm: security.selinux [ 4.030755] evm: security.ima [ 4.031522] evm: security.capability [ 4.032427] evm: HMAC attrs: 0x1 [ 4.034673] rtc_cmos 00:05: setting system clock to 2025-12-22 14:06:25 UTC (1766412385) [ 4.041067] debug: unmapping init [mem 0xffffffff89e03000-0xffffffff89ffffff] [ 4.043545] debug: unmapping init [mem 0xffffffff88b82000-0xffffffff88e58fff] [ 4.052986] Write protecting the kernel read-only data: 28672k [ 4.056277] debug: unmapping init [mem 0xffffffff87203000-0xffffffff873fffff] [ 4.058934] debug: unmapping init [mem 0xffffffff87b14000-0xffffffff87bfffff] [ 4.095571] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.105122] systemd[1]: Detected virtualization kvm. [ 4.107235] systemd[1]: Detected architecture x86-64. [ 4.109281] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.141615] systemd[1]: No hostname configured. [ 4.143338] systemd[1]: Set hostname to . [ 4.145296] random: systemd: uninitialized urandom read (16 bytes read) [ 4.147524] systemd[1]: Initializing machine ID from random generator. [ 4.189373] random: ln: uninitialized urandom read (6 bytes read) [ 4.293454] random: systemd: uninitialized urandom read (16 bytes read) [ 4.296419] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 4.301050] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 4.305528] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ OK ] Reached target Local File Systems. [ OK ] Listening on udev Kernel Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Timers. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Sockets. Starting Create Volatile Files and Directories... Starting Journal Service... Starting Setup Virtual Console... Starting Apply Kernel Variables... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.929193] device-mapper: uevent: version 1.0.3 [ 4.931654] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] [ 5.657635] random: fast init done Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.731676] virtio_net virtio0 ens2: renamed from eth0 [ 5.740622] scsi host0: ata_piix [ 5.749120] scsi host1: ata_piix [ 5.750934] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.753564] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.617065] dracut-initqueue[587]: RTNETLINK answers: File exists [ 10.541368] random: crng init done [ 10.542876] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 11.104700] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Sockets. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.363519] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.669114] SELinux: Disabled at runtime. [ 12.728277] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.736617] systemd[1]: Detected virtualization kvm. [ 12.738508] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 13.288718] systemd[1]: initrd-switch-root.service: Succeeded. [ 13.291763] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 13.298735] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 13.302154] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 13.305232] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 13.312251] systemd[1]: Starting Journal Service... Starting Journal Service... [ 13.317640] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Switch Root. [ OK ] Listening on Process Core Dump Socket. Mounting Huge Pages File System... [ OK ] Listening on udev Control Socket. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Mounting POSIX Message Queue File System... [ OK ] Created slice system-sshd\x2dkeygen.slice. Activating swap /dev/disk/by-label/SWAP... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target RPC Port Mapper. [ 13.428589] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Apply Kernel Variables... [ OK ] Stopped target Initrd File Systems. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... Mounting Kernel Debug File System... Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-getty.slice. [ OK ] Stopped target Initrd Root File System. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 13.836654] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 14.208046] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 14.226340] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 14.342569] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 14.366881] EDAC sbridge: Ver: 1.1.2 [ 16.172561] Key type dns_resolver registered [ 16.479412] NFS: Registering the id_resolver key type [ 16.481121] Key type id_resolver registered [ 16.482417] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting Network Manager Wait Online... [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg445-server login: [ 66.646901] hrtimer: interrupt took 5889664 ns [ 81.322650] libcfs: loading out-of-tree module taints kernel. [ 81.350158] Key type ._llcrypt registered [ 81.352014] Key type .llcrypt registered [ 81.445303] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_hostid [ 94.886482] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 95.847849] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 95.861273] alg: No test for adler32 (adler32-zlib) [ 97.083643] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 97.688230] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 99.367169] Key type lgssc registered [ 100.831985] Lustre: Echo OBD driver; http://www.lustre.org/ [ 114.689653] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 115.494106] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 123.350032] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 131.108209] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 138.487745] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 152.999144] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 164.343882] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 164.386393] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 164.400803] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 165.606807] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 165.642147] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 165.721932] Lustre: lustre-MDT0000: new disk, initializing [ 165.832231] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 165.854992] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 169.244332] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 181.106490] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 181.180858] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 181.289315] Lustre: 6516:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 181.321739] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 181.325497] Lustre: Skipped 1 previous similar message [ 181.430989] Lustre: lustre-MDT0001: new disk, initializing [ 181.511886] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 181.550949] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 181.554258] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 184.834723] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 189.088686] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 196.546855] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 196.603803] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 196.813512] Lustre: lustre-OST0000: new disk, initializing [ 196.816372] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 196.874900] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 200.994664] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 202.223602] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 202.232810] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 202.265250] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 211.752880] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 211.845339] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 211.923715] Lustre: lustre-OST0001: new disk, initializing [ 211.926270] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 212.001831] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 216.039792] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 221.722939] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 221.728310] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 221.765769] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 224.713607] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 229.255604] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 239.199459] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing check_logdir /tmp/testlogs/ [ 242.977724] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing yml_node [ 247.432726] Lustre: DEBUG MARKER: Client: 2.17.0.RC3 [ 249.794366] Lustre: DEBUG MARKER: MDS: 2.17.0.RC3 [ 252.311850] Lustre: DEBUG MARKER: OSS: 2.17.0.RC3 [ 254.301146] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Dec 22 09:10:35 EST 2025 [ 274.443873] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 276.016362] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 293.344385] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 293.348768] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 293.376670] Lustre: Skipped 1 previous similar message [ 293.405714] Lustre: Skipped 3 previous similar messages [ 297.912422] LustreError: 12226:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 298.035979] Lustre: server umount lustre-MDT0000 complete [ 301.416652] LustreError: 6506:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766412682 with bad export cookie 17174548002924521474 [ 301.421461] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 301.437972] LustreError: 6506:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 303.583839] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 303.587484] LustreError: 6521:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 303.607952] Lustre: Skipped 2 previous similar messages [ 303.646098] LustreError: 6521:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 303.651137] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 306.599351] LustreError: 6522:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 306.602924] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 306.616348] Lustre: Skipped 1 previous similar message [ 307.634500] LustreError: 12427:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 307.636700] LustreError: 12427:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 307.805938] Lustre: server umount lustre-MDT0001 complete [ 310.975092] LustreError: 12630:0:(obd_class.h:479:obd_check_dev()) Device 20 not setup [ 310.977824] LustreError: 12630:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 311.091207] Lustre: server umount lustre-OST0000 complete [ 313.900232] LustreError: 12830:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 313.906045] LustreError: 12830:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 314.059638] Lustre: server umount lustre-OST0001 complete [ 319.553869] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_hostid [ 326.489454] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 335.248606] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 340.872386] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 346.103285] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 351.371995] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 357.086694] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 357.143556] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 357.316985] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 357.353792] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 357.405120] Lustre: lustre-MDT0000: new disk, initializing [ 357.499534] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 357.527504] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 360.855571] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 369.677301] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 369.758804] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 369.849795] Lustre: 15015:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 369.910906] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 369.915041] Lustre: Skipped 1 previous similar message [ 370.008429] Lustre: lustre-MDT0001: new disk, initializing [ 370.126506] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 370.167137] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 370.175822] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 373.935882] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 380.741628] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 384.154708] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 390.511521] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 390.610268] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 390.843630] Lustre: lustre-OST0000: new disk, initializing [ 390.852575] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 390.949314] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 392.690892] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 392.695918] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 392.844638] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 396.091207] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 403.304520] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 406.862569] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 407.058697] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 411.164673] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 411.435913] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 413.664617] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 413.667243] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 413.689699] Lustre: Skipped 1 previous similar message [ 413.693801] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 418.273453] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 418.276538] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 418.285201] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 418.302838] Lustre: Skipped 1 previous similar message [ 419.396303] LustreError: 17955:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 419.404670] LustreError: 17955:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 419.511832] Lustre: server umount lustre-OST0000 complete [ 423.392814] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 423.401384] Lustre: Skipped 1 previous similar message [ 428.512976] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 428.521277] Lustre: Skipped 3 previous similar messages [ 429.224511] LustreError: 18156:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 429.227656] LustreError: 18156:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 429.322728] Lustre: server umount lustre-MDT0000 complete [ 432.201816] LustreError: 16761:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766412813 with bad export cookie 17174548002924523007 [ 432.205588] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 432.214087] LustreError: 16761:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 432.492187] Lustre: server umount lustre-MDT0001 complete [ 436.464404] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 09:13:38 (1766412818) [ 441.586995] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 441.939026] LustreError: 18884:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 441.947880] LustreError: 18884:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 442.008718] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 445.348537] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 447.459438] LustreError: 18884:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 451.553307] LustreError: 18884:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 453.135923] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 457.406556] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 462.513724] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 465.976046] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 471.146483] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 475.867475] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 480.653576] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 494.602245] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 499.680723] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 499.689636] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 499.697733] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 505.693467] LustreError: 21210:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 505.695814] LustreError: 21210:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 505.825906] Lustre: server umount lustre-MDT0000 complete [ 509.373898] LustreError: 18863:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766412890 with bad export cookie 17174548002924523889 [ 509.382534] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 509.396577] LustreError: 18863:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 509.837114] Lustre: server umount lustre-MDT0001 complete [ 524.224582] Lustre: server umount lustre-OST0000 complete [ 541.218714] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 550.666915] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 550.967931] LustreError: 22939:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 551.034538] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 551.038312] Lustre: Skipped 2 previous similar messages [ 555.098612] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 563.873670] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 568.787388] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 571.774666] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 579.036649] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 585.058817] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 593.265277] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 593.367399] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 593.590496] Lustre: lustre-OST0001: new disk, initializing [ 593.602172] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 594.900273] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 594.923235] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 595.048585] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 598.230498] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 606.122421] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 609.222936] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 625.635915] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 625.643686] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 625.663390] Lustre: Skipped 2 previous similar messages [ 625.669717] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 625.673959] Lustre: Skipped 6 previous similar messages [ 631.005699] LustreError: 26424:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 631.010616] LustreError: 26424:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 631.779316] LustreError: 22938:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 631.794026] LustreError: 22938:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 633.097942] Lustre: server umount lustre-MDT0000 complete [ 636.922055] LustreError: 22919:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766413018 with bad export cookie 17174548002924524736 [ 636.925720] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 636.935208] LustreError: 22919:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 637.249543] Lustre: server umount lustre-MDT0001 complete [ 651.348671] Lustre: server umount lustre-OST0000 complete [ 665.451076] Lustre: server umount lustre-OST0001 complete [ 680.949153] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 692.395625] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 692.828352] LustreError: 28205:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 692.840148] LustreError: 28205:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 692.925762] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 692.929203] Lustre: Skipped 3 previous similar messages [ 697.687620] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 708.518940] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 713.555956] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 716.412352] Lustre: 29321:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 723.330329] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 729.132623] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 737.631879] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 743.818617] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 751.075515] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 754.573818] Lustre: 31176:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 770.020400] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 770.024846] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 770.085389] Lustre: Skipped 6 previous similar messages [ 770.089096] Lustre: Skipped 6 previous similar messages [ 771.834594] LustreError: 31377:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 771.837630] LustreError: 31377:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 772.006429] Lustre: server umount lustre-MDT0000 complete [ 775.141846] LustreError: 28200:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 775.173493] LustreError: 28200:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 776.460865] LustreError: 28186:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766413157 with bad export cookie 17174548002924526031 [ 776.467498] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 776.470501] LustreError: 28186:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 776.851618] Lustre: server umount lustre-MDT0001 complete [ 791.166059] Lustre: server umount lustre-OST0000 complete [ 805.573561] Lustre: server umount lustre-OST0001 complete [ 813.526685] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 09:19:55 (1766413195) [ 839.591737] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 850.153478] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 855.261523] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 863.697649] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 867.976505] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 870.774367] Lustre: 35077:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 877.055536] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 883.019827] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 891.847669] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 897.998297] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 905.193915] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 908.652698] Lustre: 36935:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 920.544429] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 920.554333] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 920.591063] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 920.609315] Lustre: Skipped 1 previous similar message [ 924.339192] LustreError: 37087:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 924.342206] LustreError: 37087:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 924.450696] Lustre: server umount lustre-MDT0000 complete [ 926.700048] LustreError: 35457:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 926.712145] LustreError: 35457:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 927.311445] LustreError: 33942:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766413308 with bad export cookie 17174548002924527137 [ 927.316802] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 927.323966] LustreError: 33942:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 927.692969] Lustre: server umount lustre-MDT0001 complete [ 941.105622] Lustre: server umount lustre-OST0000 complete [ 954.976853] Lustre: server umount lustre-OST0001 complete [ 963.028230] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 09:22:24 (1766413344) [ 977.132067] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 986.521896] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 987.163035] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 987.174199] Lustre: Skipped 7 previous similar messages [ 990.606519] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 999.093532] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1003.594269] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 1006.447711] Lustre: 40178:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1012.485340] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1017.717238] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 1026.037968] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1031.556596] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 1038.824756] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1042.080849] Lustre: 42031:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1048.017437] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1048.020014] Lustre: Skipped 1 previous similar message [ 1052.647225] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1052.677502] Lustre: Skipped 6 previous similar messages [ 1057.415065] Lustre: server umount lustre-MDT0000 complete [ 1061.035357] LustreError: 39041:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766413442 with bad export cookie 17174548002924528243 [ 1061.041389] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1061.055907] LustreError: 39041:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1067.498623] Lustre: server umount lustre-MDT0001 complete [ 1075.066123] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1086.431976] LustreError: 3654:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff88c3c65df800 x1852217731968000/t0(0) o250->MGC192.168.204.145@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1089.920052] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 1095.996489] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1097.617125] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1102.189875] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 1103.344525] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1103.350916] Lustre: Skipped 1 previous similar message [ 1106.884352] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1108.091240] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1110.933635] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1112.156980] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1118.694864] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1121.392354] Lustre: server umount lustre-MDT0000 complete [ 1124.885633] LustreError: 44348:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766413506 with bad export cookie 17174548002924529433 [ 1124.888084] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1124.919729] LustreError: 44348:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1125.191019] Lustre: server umount lustre-MDT0001 complete [ 1138.320882] Lustre: server umount lustre-OST0000 complete [ 1152.471483] Lustre: server umount lustre-OST0001 complete [ 1161.827076] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 09:25:43 (1766413543) [ 1175.586544] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 1183.298926] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1183.672366] LustreError: 46410:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1183.689043] LustreError: 46410:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 25 previous similar messages [ 1187.276381] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 1194.760665] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1198.920778] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 1201.428383] Lustre: 47530:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1206.934637] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1211.911343] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 1219.096115] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1224.351580] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 1230.083138] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1238.454250] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1249.259796] LustreError: 49608:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 1249.262320] LustreError: 49608:0:(obd_class.h:479:obd_check_dev()) Skipped 67 previous similar messages [ 1249.306631] Lustre: server umount lustre-MDT0000 complete [ 1251.296019] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1252.092040] LustreError: 49391:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766413633 with bad export cookie 17174548002924530217 [ 1252.095001] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1252.113665] LustreError: 49391:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1252.397552] Lustre: server umount lustre-MDT0001 complete [ 1264.705280] Lustre: server umount lustre-OST0000 complete [ 1278.524521] Lustre: server umount lustre-OST0001 complete [ 1286.412712] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 09:27:48 (1766413668) [ 1288.610934] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1290.022636] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 09:27:51 (1766413671) [ 1298.744526] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_hostid [ 1305.757604] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 1313.730200] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1318.871180] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1324.616456] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1324.694882] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1325.009973] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1325.090366] Lustre: lustre-MDT0000: new disk, initializing [ 1325.174816] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1328.546973] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 1335.054596] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1340.981497] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1341.062196] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1341.215935] Lustre: lustre-OST0000: new disk, initializing [ 1341.220463] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1341.228881] Lustre: Skipped 1 previous similar message [ 1343.126136] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1343.132645] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1343.159672] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1346.450508] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 1352.768345] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1354.816228] Lustre: DEBUG MARKER: create test files [ 1378.789962] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1378.805758] Lustre: Skipped 10 previous similar messages [ 1378.814407] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1378.819965] Lustre: Skipped 19 previous similar messages [ 1379.677244] Lustre: server umount lustre-OST0000 complete [ 1382.631781] Lustre: server umount lustre-MDT0000 complete [ 1391.407614] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 1393.762597] Key type lgssc unregistered [ 1394.066622] LNet: 54947:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1394.073344] LNetError: 54947:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1395.109884] LNet: Removed LNI 192.168.204.145@tcp [ 1395.870170] Key type .llcrypt unregistered [ 1395.871921] Key type ._llcrypt unregistered [ 1422.928137] Key type ._llcrypt registered [ 1422.930246] Key type .llcrypt registered [ 1423.030519] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 1424.351254] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1424.386476] alg: No test for adler32 (adler32-zlib) [ 1425.609749] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 1425.871828] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 1427.511140] Key type lgssc registered [ 1428.808387] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1436.054338] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1436.080544] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1437.570717] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1440.709683] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 1445.625922] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1451.967319] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1452.269815] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1458.219451] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 1462.522885] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1462.609286] Lustre: 57558:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1462.616237] Lustre: 57558:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1462.627518] Lustre: 57558:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1462.634935] Lustre: 57558:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1462.651467] Lustre: 57558:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1462.654278] Lustre: 57558:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1463.682054] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1469.055596] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1474.284748] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1508.832716] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1508.853901] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1511.880668] LustreError: 58329:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1512.019260] Lustre: server umount lustre-OST0000 complete [ 1515.354119] LustreError: 58531:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1515.358337] LustreError: 58531:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1515.581884] Lustre: server umount lustre-MDT0000 complete [ 1524.956587] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 1527.661115] Key type lgssc unregistered [ 1527.962672] LNet: 59065:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1527.971701] LNetError: 59065:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1529.006890] LNet: Removed LNI 192.168.204.145@tcp [ 1529.924774] Key type .llcrypt unregistered [ 1529.925950] Key type ._llcrypt unregistered [ 1559.798490] Key type ._llcrypt registered [ 1559.799858] Key type .llcrypt registered [ 1559.901766] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 1560.725285] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1560.886487] alg: No test for adler32 (adler32-zlib) [ 1561.944692] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 1562.124419] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 1563.768093] Key type lgssc registered [ 1564.991039] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1572.576683] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1572.616307] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1574.266589] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1578.560604] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 1584.724609] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1592.310150] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1592.844882] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1598.630682] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 1598.972180] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1599.029707] Lustre: 61772:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1599.034355] Lustre: 61772:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1599.038841] Lustre: 61772:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1599.042995] Lustre: 61772:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1599.049365] Lustre: 61772:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1599.054759] Lustre: 61772:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1603.943710] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1609.480753] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1650.144546] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1650.159928] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1650.189057] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1653.009222] LustreError: 62492:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1653.127506] Lustre: server umount lustre-OST0000 complete [ 1656.542814] LustreError: 62696:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1656.545842] LustreError: 62696:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1656.755719] Lustre: server umount lustre-MDT0000 complete [ 1666.607494] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 1669.778706] Key type lgssc unregistered [ 1670.101507] LNet: 63226:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1670.105470] LNetError: 63226:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1671.141674] LNet: Removed LNI 192.168.204.145@tcp [ 1672.005662] Key type .llcrypt unregistered [ 1672.007463] Key type ._llcrypt unregistered [ 1688.274471] Key type ._llcrypt registered [ 1688.276374] Key type .llcrypt registered [ 1688.369852] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_hostid [ 1701.381778] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 1702.112385] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1702.138119] alg: No test for adler32 (adler32-zlib) [ 1703.266878] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 1703.495249] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 1705.183204] Key type lgssc registered [ 1706.167890] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1715.156560] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1721.781557] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1729.059785] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1735.995967] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1743.321766] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1743.397159] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1743.418983] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1744.699768] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1744.743942] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1744.866988] Lustre: lustre-MDT0000: new disk, initializing [ 1744.992120] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1745.011317] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1748.676531] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 1758.573882] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1758.632194] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1758.699566] Lustre: 66354:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1758.761786] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1758.764216] Lustre: Skipped 1 previous similar message [ 1758.826849] Lustre: lustre-MDT0001: new disk, initializing [ 1758.922800] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1758.959337] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1758.975865] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1762.649239] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 1769.288726] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1772.726076] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1779.456936] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1779.546978] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1779.860236] Lustre: lustre-OST0000: new disk, initializing [ 1779.864381] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1780.008921] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1786.050673] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 1788.508404] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1788.517089] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1788.616927] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1793.609918] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1798.325793] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1798.585933] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1803.612354] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1803.981855] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1809.376435] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1809.379624] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1809.387368] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1809.399865] Lustre: Skipped 1 previous similar message [ 1812.563417] LustreError: 69295:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1812.777301] Lustre: server umount lustre-OST0000 complete [ 1819.620155] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1819.620782] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1819.624278] Lustre: Skipped 1 previous similar message [ 1819.627208] Lustre: Skipped 1 previous similar message [ 1824.738921] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1824.741439] Lustre: Skipped 1 previous similar message [ 1828.833680] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1828.839507] Lustre: Skipped 1 previous similar message [ 1831.391156] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1831.525843] LustreError: 69497:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1831.528278] LustreError: 69497:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1831.613359] Lustre: server umount lustre-MDT0000 complete [ 1834.706672] LustreError: 66348:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766414216 with bad export cookie 8557801918475761814 [ 1834.713368] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1834.733580] LustreError: 66348:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1834.881490] LustreError: 69699:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1834.884889] LustreError: 69699:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1835.083612] Lustre: server umount lustre-MDT0001 complete [ 1842.331593] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 09:37:04 (1766414224) [ 1852.081988] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 1865.506665] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1888.673945] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_hostid [ 1897.471969] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 1906.731700] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1912.440962] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1918.829450] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1924.654452] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1931.116856] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1931.172287] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1931.337788] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1931.361854] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1931.422844] Lustre: lustre-MDT0000: new disk, initializing [ 1931.510277] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1931.529536] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1935.076318] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 1944.887136] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1944.966032] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1945.079120] Lustre: 73938:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1945.152068] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1945.154754] Lustre: Skipped 1 previous similar message [ 1945.220291] Lustre: lustre-MDT0001: new disk, initializing [ 1945.316749] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1945.387866] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1945.407797] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1949.353625] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 1956.015376] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1959.798473] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1966.331468] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1966.416705] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1966.790384] Lustre: lustre-OST0000: new disk, initializing [ 1966.803198] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1966.878150] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1972.452756] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 1973.311391] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1973.321263] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1973.451352] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1979.433798] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1983.595270] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1983.796555] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1987.977566] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1988.208221] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1994.224276] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1994.226204] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1994.233261] LustreError: Skipped 1 previous similar message [ 1994.233965] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1994.246044] Lustre: Skipped 1 previous similar message [ 1996.344455] LustreError: 76879:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1996.356642] LustreError: 76879:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1996.475822] Lustre: server umount lustre-OST0000 complete [ 2001.888587] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2001.899885] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2004.447785] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2004.450627] Lustre: Skipped 2 previous similar messages [ 2014.175968] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2014.295707] LustreError: 77080:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2014.307797] LustreError: 77080:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2014.387986] Lustre: server umount lustre-MDT0000 complete [ 2018.243531] LustreError: 73929:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766414399 with bad export cookie 8557801918475762696 [ 2018.257176] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2018.588518] Lustre: server umount lustre-MDT0001 complete [ 2026.733572] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 09:40:08 (1766414408) [ 2033.336135] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2033.807226] LustreError: 77954:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2033.827429] LustreError: 77954:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2033.873737] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2037.266419] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 2039.266968] LustreError: 77954:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2044.388849] LustreError: 77954:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2044.948404] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2048.460635] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 2052.895061] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2055.408452] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2060.236360] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2065.238510] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 2069.501503] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2072.092812] Lustre: 80292:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting 5e2c8c65-1048-4383-82a9-c54a63f66127 at adminstrative request [ 2075.108990] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2075.115822] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2085.352804] Lustre: MGS: Client 5e2c8c65-1048-4383-82a9-c54a63f66127 (at 0@lo) reconnecting [ 2085.356306] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x76c36b80f0e77f73 to 0x76c36b80f0e78213 [ 2085.368585] Lustre: MGC192.168.204.145@tcp: Connection restored to 0@lo (at 0@lo) [ 2108.710989] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2108.771320] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2108.876731] Lustre: lustre-OST0001: new disk, initializing [ 2108.884126] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2108.976722] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2108.987276] Lustre: Skipped 2 previous similar messages [ 2113.743469] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 2118.811958] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2118.819055] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2118.929424] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2120.502302] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2124.266760] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2124.276606] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2124.284168] Lustre: Skipped 1 previous similar message [ 2124.287346] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2124.291350] Lustre: Skipped 5 previous similar messages [ 2125.375845] LustreError: 81430:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 2125.383035] LustreError: 81430:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 2125.457125] Lustre: server umount lustre-OST0001 complete [ 2129.377444] LustreError: 80110:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2129.390564] LustreError: 80110:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2134.503714] LustreError: 79551:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2134.531413] LustreError: 79551:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2135.273510] Lustre: server umount lustre-OST0000 complete [ 2139.619710] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2139.636059] Lustre: Skipped 4 previous similar messages [ 2144.770606] LustreError: 81832:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 2144.777540] LustreError: 81832:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 2144.913782] Lustre: server umount lustre-MDT0000 complete [ 2148.555744] LustreError: 77933:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766414530 with bad export cookie 8557801918475764243 [ 2148.563493] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2148.575354] LustreError: 77933:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 2148.898069] Lustre: server umount lustre-MDT0001 complete [ 2156.421987] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 09:42:17 (1766414537) [ 2157.841372] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2159.809089] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 09:42:21 (1766414541) [ 2161.333594] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2163.212575] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 09:42:24 (1766414544) [ 2164.979160] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2166.722264] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 09:42:28 (1766414548) [ 2168.391271] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2170.078538] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 09:42:31 (1766414551) [ 2171.697188] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2176.985682] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2178.097466] loop: module loaded [ 2178.126093] loop0: detected capacity change from 0 to 4294967296 [ 2178.184298] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2179.855467] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2183.201639] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2187.089780] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2194.816504] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2198.787950] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2202.445193] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2205.952051] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2212.394602] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2212.513728] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2212.542759] LustreError: 84704:0:(obd_config.c:783:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2212.546554] LustreError: 84704:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2212.552478] LustreError: 84704:0:(tgt_mount.c:2447:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2212.557964] LustreError: 84704:0:(super25.c:199:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2229.309278] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_hostid [ 2237.743088] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 2246.793392] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2254.349301] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2262.632847] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2270.406988] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2277.648693] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2277.729648] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2277.968740] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2278.025676] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2278.113925] Lustre: lustre-MDT0000: new disk, initializing [ 2278.199030] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2278.226187] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2281.670755] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 2291.851528] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2291.931757] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2292.065181] Lustre: 87548:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2292.454388] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2296.647209] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 2303.580766] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2306.522273] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2312.606369] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2312.672546] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2318.803035] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 2323.011784] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2323.131182] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2325.714982] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2333.855834] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2334.041123] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2337.412534] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2337.635121] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2343.907362] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2343.910039] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2343.914066] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2343.914075] Lustre: Skipped 8 previous similar messages [ 2343.922195] Lustre: Skipped 1 previous similar message [ 2343.930514] LustreError: Skipped 1 previous similar message [ 2345.421584] LustreError: 90493:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2345.432672] LustreError: 90493:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2345.610069] Lustre: server umount lustre-OST0000 complete [ 2354.146498] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2354.159609] LustreError: Skipped 1 previous similar message [ 2355.598882] Lustre: server umount lustre-MDT0000 complete [ 2359.147919] LustreError: 87542:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766414740 with bad export cookie 8557801918475764838 [ 2359.150363] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2359.159371] LustreError: 87542:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2359.464223] Lustre: server umount lustre-MDT0001 complete [ 2366.278944] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 09:45:47 (1766414747) [ 2372.076691] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2372.409513] LustreError: 91567:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2376.149675] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 2383.724948] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2387.482685] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 2392.044987] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2395.006479] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2400.800388] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2406.127223] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 2411.419126] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2415.173258] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2418.582776] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 3 sec [ 2422.266606] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2422.455801] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2427.899354] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2427.977278] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2428.084369] Lustre: lustre-OST0001: new disk, initializing [ 2428.086314] Lustre: Skipped 2 previous similar messages [ 2428.090478] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2428.096826] Lustre: Skipped 3 previous similar messages [ 2428.170963] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2428.178505] Lustre: Skipped 5 previous similar messages [ 2433.301587] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 2437.195229] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2437.207955] Lustre: Skipped 2 previous similar messages [ 2437.217255] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2437.268282] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2439.752738] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2443.813079] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2444.026585] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2447.373637] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2447.520457] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2463.423619] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2486.323025] Lustre: Failing over lustre-MDT0000 [ 2486.753975] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2486.763490] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2486.789495] Lustre: Skipped 2 previous similar messages [ 2486.805387] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2486.822962] Lustre: Skipped 3 previous similar messages [ 2486.845539] LustreError: 96498:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 2486.848554] LustreError: 96498:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2486.999240] Lustre: server umount lustre-MDT0000 complete [ 2488.813700] LustreError: 91566:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2488.837194] LustreError: 91566:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 2505.183330] Lustre: 64564:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1766414870/real 1766414870] req@0000000021426101 x1852219414793472/t0(0) o400->MGC192.168.204.145@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1766414886 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2505.197817] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2511.441095] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2511.443218] LDISKFS-fs (dm-0): recovery complete [ 2511.458873] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2515.615783] LustreError: 97216:0:(import.c:337:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 2515.636437] LustreError: 97216:0:(import.c:361:ptlrpc_invalidate_import()) @@@ still on sending list req@000000002ecb22f0 x1852219414799232/t0(0) o250->MGC192.168.204.145@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1766414896 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2515.664943] LustreError: 97216:0:(import.c:371:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 2516.453961] LustreError: 64562:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@000000000d39f7f8 x1852219414802944/t0(0) o250->MGC192.168.204.145@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2516.803817] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2516.974884] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2516.977108] Lustre: 96352:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2520.988649] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 2522.097266] Lustre: 92280:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2522.114448] Lustre: 92280:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 1 previous similar message [ 2522.136634] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2522.212653] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2522.543067] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2522.898133] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2522.922569] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 2523.247606] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2523.266072] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 2523.281199] Lustre: 97248:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2523.309590] Lustre: 97248:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 7 previous similar messages [ 2523.969479] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2523.980811] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 2 previous similar messages [ 2524.295133] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2524.303118] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 2 previous similar messages [ 2525.598830] Lustre: 97248:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2525.612053] Lustre: 97248:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 6 previous similar messages [ 2526.268655] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2526.278208] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 6 previous similar messages [ 2526.608043] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2526.610909] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 6 previous similar messages [ 2529.797071] Lustre: 97248:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2529.805827] Lustre: 97248:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 12 previous similar messages [ 2530.448817] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2530.458064] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 2530.791139] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2530.796223] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 2538.107806] Lustre: 97248:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2538.122597] Lustre: 97248:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 25 previous similar messages [ 2538.469848] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2538.478434] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 24 previous similar messages [ 2538.799639] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2538.804991] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 24 previous similar messages [ 2554.406633] Lustre: 97248:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2554.417247] Lustre: 97248:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 50 previous similar messages [ 2554.736514] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2554.741593] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 50 previous similar messages [ 2555.079550] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2555.083195] LustreError: 97248:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 50 previous similar messages [ 2576.163330] Lustre: 97248:0:(ldlm_lib.c:2377:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2576.167089] Lustre: 97248:0:(ldlm_lib.c:2387:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2576.174360] Lustre: 97248:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 02f3225a-3c72-4bb1-b1d6-024f1e71a217@192.168.204.45@tcp [ 2576.181771] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2576.185647] Lustre: 97248:0:(ldlm_lib.c:1897:abort_req_replay_queue()) @@@ aborted: req@00000000c52746ca x1852219403076608/t0(8589934765) o101->02f3225a-3c72-4bb1-b1d6-024f1e71a217@192.168.204.45@tcp:358/0 lens 592/0 e 0 to 0 dl 1766414968 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2576.204856] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2576.217447] Lustre: lustre-MDT0000: Denying connection for new client 02f3225a-3c72-4bb1-b1d6-024f1e71a217 (at 192.168.204.45@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2576.224539] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2576.272259] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2576.274375] Lustre: Skipped 2 previous similar messages [ 2576.279892] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2576.315578] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:44 to 0x2c0000401:545) [ 2576.319654] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:76 to 0x280000401:577) [ 2592.226130] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2592.230092] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2592.243334] Lustre: Skipped 3 previous similar messages [ 2603.999819] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2604.093359] Lustre: server umount lustre-OST0000 complete [ 2607.025402] LustreError: 93164:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2607.048100] LustreError: 93164:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 41 previous similar messages [ 2617.325453] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2617.331197] Lustre: Skipped 12 previous similar messages [ 2621.407243] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2621.460746] LustreError: 98114:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 2621.467741] LustreError: 98114:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 2621.566394] Lustre: server umount lustre-OST0001 complete [ 2639.327156] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2639.462755] Lustre: server umount lustre-MDT0000 complete [ 2642.320307] LustreError: 94603:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766415023 with bad export cookie 8557801918475816057 [ 2642.322244] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2642.327797] LustreError: 94603:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2642.667076] Lustre: server umount lustre-MDT0001 complete [ 2650.187195] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 09:50:31 (1766415031) [ 2656.297853] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2656.304528] Lustre: *** cfs_fail_loc=197, val=0*** [ 2671.840706] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2676.959490] LustreError: 99213:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.145@tcp: failed processing log, type 4: rc = -110 [ 2682.108735] Lustre: *** cfs_fail_loc=197, val=0*** [ 2682.110410] Lustre: Skipped 65 previous similar messages [ 2702.687297] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2702.693109] Lustre: Skipped 1 previous similar message [ 2707.081646] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 2711.512939] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2713.776332] Lustre: server umount lustre-OST0000 complete [ 2721.800522] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 09:51:43 (1766415103) [ 2741.549992] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2741.927930] LustreError: 100722:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2741.951499] LustreError: 100722:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 2745.605455] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 2751.653993] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2754.979682] LustreError: 100752:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2754.993523] LustreError: 100752:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2755.001987] LustreError: 100752:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 13, retries 0, failed: rc = -5 [ 2755.305601] Lustre: server umount lustre-MDT0000 complete [ 2768.745958] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2784.648389] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_hostid [ 2792.969345] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 2800.591453] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2806.041150] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2811.401257] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2816.764262] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2822.468473] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2822.556474] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2822.727742] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2822.759041] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2822.812965] Lustre: lustre-MDT0000: new disk, initializing [ 2822.902313] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2826.708537] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 2835.478096] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2835.538178] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2835.594343] Lustre: 104793:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2835.895792] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2838.861917] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 2844.795569] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2847.705954] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2852.979690] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2853.049247] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2857.989474] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 2860.213585] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2864.542907] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2867.769975] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2867.980084] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2871.268922] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2871.442105] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2875.872497] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2875.875779] LustreError: Skipped 2 previous similar messages [ 2875.881136] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2875.905764] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2875.905771] Lustre: Skipped 6 previous similar messages [ 2875.915666] Lustre: Skipped 6 previous similar messages [ 2879.163839] LustreError: 107736:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2879.175306] LustreError: 107736:0:(obd_class.h:479:obd_check_dev()) Skipped 37 previous similar messages [ 2879.268216] Lustre: server umount lustre-OST0000 complete [ 2896.353424] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2896.527337] Lustre: server umount lustre-MDT0000 complete [ 2899.605289] LustreError: 104785:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766415281 with bad export cookie 8557801918475826571 [ 2899.606203] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2899.618206] LustreError: 104785:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2899.905672] Lustre: server umount lustre-MDT0001 complete [ 2907.383431] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 09:54:48 (1766415288) [ 2913.976597] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 2916.435666] Key type lgssc unregistered [ 2916.666482] LNet: 108867:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2916.674788] LNetError: 108867:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2916.687924] LNet: Removed LNI 192.168.204.145@tcp [ 2917.346178] Key type .llcrypt unregistered [ 2917.348119] Key type ._llcrypt unregistered [ 2931.316908] Key type ._llcrypt registered [ 2931.318371] Key type .llcrypt registered [ 2931.411974] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_hostid [ 2945.631607] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 2947.032491] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2947.058401] alg: No test for adler32 (adler32-zlib) [ 2948.190095] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 2948.391907] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 2950.103167] Key type lgssc registered [ 2951.111766] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2958.540629] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2964.061106] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2969.236914] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2974.937058] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2979.366980] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2984.923935] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2984.972411] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2984.983625] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2986.253990] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2986.291466] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2986.392460] Lustre: lustre-MDT0000: new disk, initializing [ 2986.534146] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2986.557612] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2990.131774] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 2999.356390] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2999.420535] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2999.614819] Lustre: lustre-OST0000: new disk, initializing [ 2999.619412] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2999.630133] Lustre: Skipped 1 previous similar message [ 2999.705549] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3004.477179] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 3009.108324] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3009.144189] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3009.176502] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3013.204526] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3013.263733] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3013.361479] Lustre: lustre-OST0001: new disk, initializing [ 3013.364514] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3013.425591] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3014.945120] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3014.953828] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3014.982489] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3017.755671] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 3030.498497] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3030.506035] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3030.515711] Lustre: Skipped 1 previous similar message [ 3031.960843] LustreError: 114441:0:(obd_class.h:479:obd_check_dev()) Device 17 not setup [ 3032.073955] Lustre: server umount lustre-MDT0000 complete [ 3037.654652] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3041.398551] LustreError: 112202:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766415422 with bad export cookie 16137264387819362404 [ 3041.411302] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3041.413520] LustreError: 112202:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3041.447115] LustreError: 115037:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 3041.458100] LustreError: 115037:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3041.517377] Lustre: server umount lustre-OST0000 complete [ 3044.427549] LustreError: 115238:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 3044.430817] LustreError: 115238:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 3044.552775] Lustre: server umount lustre-OST0001 complete [ 3051.831627] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 09:57:13 (1766415433) [ 3059.553563] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 3066.402164] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3067.325905] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3071.927144] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3072.838043] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3078.505093] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 09:57:40 (1766415460) [ 3090.471077] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3092.397965] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3094.037500] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3095.738759] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3096.807580] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3098.863667] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3113.996878] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_hostid [ 3122.819958] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 3130.547868] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3135.615887] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3140.936619] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3146.333527] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3152.476500] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3152.558534] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3152.870621] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3152.904409] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3152.980873] Lustre: lustre-MDT0000: new disk, initializing [ 3153.104712] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3153.123788] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3156.903338] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 3164.238494] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3164.285386] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3164.341627] Lustre: 120566:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3164.396831] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3164.399710] Lustre: Skipped 1 previous similar message [ 3164.456545] Lustre: lustre-MDT0001: new disk, initializing [ 3164.531927] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3164.577338] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3164.594147] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3167.699474] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 3173.575852] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3176.246342] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3181.133854] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3181.191963] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3181.389776] Lustre: lustre-OST0000: new disk, initializing [ 3181.398181] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3181.476581] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3185.968341] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 3188.790820] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3188.796634] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3188.857549] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3191.372948] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3194.586178] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3194.734493] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3197.404972] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3197.582852] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3199.460560] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3199.462350] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3199.463371] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3199.463376] Lustre: Skipped 1 previous similar message [ 3199.470296] Lustre: Skipped 1 previous similar message [ 3204.586997] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3204.591613] Lustre: Skipped 2 previous similar messages [ 3205.174300] LustreError: 123504:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3205.183254] LustreError: 123504:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3205.261240] Lustre: server umount lustre-OST0000 complete [ 3209.709174] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3209.716967] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3209.719831] Lustre: Skipped 1 previous similar message [ 3214.045226] LustreError: 123706:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 3214.047691] LustreError: 123706:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3214.136266] Lustre: server umount lustre-MDT0000 complete [ 3216.415527] LustreError: 121345:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766415597 with bad export cookie 16137264387819363629 [ 3216.416352] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3216.423346] LustreError: 121345:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3216.626238] Lustre: server umount lustre-MDT0001 complete [ 3222.076223] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 10:00:04 (1766415604) [ 3226.795675] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3227.128460] LustreError: 124579:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3227.139483] LustreError: 124579:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3227.217819] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3229.956547] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 3232.224872] LustreError: 124579:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3237.001154] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3240.704381] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 3245.288022] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3248.327318] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3253.780462] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3258.295104] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 3262.724617] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3277.249269] Lustre: *** cfs_fail_loc=159, val=0*** [ 3277.252924] Lustre: Skipped 1 previous similar message [ 3293.714171] Lustre: lustre-MDT0000: Client bbb5ae04-8546-498c-be7e-99c2062ca276 (at 192.168.204.45@tcp) reconnecting [ 3294.822331] Lustre: *** cfs_fail_loc=159, val=0*** [ 3294.824948] Lustre: Skipped 5 previous similar messages [ 3310.104305] Lustre: lustre-MDT0000: Client bbb5ae04-8546-498c-be7e-99c2062ca276 (at 192.168.204.45@tcp) reconnecting [ 3313.636246] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3313.642371] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3313.651710] Lustre: Skipped 1 previous similar message [ 3313.660464] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3314.656897] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3319.230128] LustreError: 127098:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3319.241904] LustreError: 127098:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 3319.346846] Lustre: server umount lustre-OST0000 complete [ 3324.384690] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3324.393805] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3324.402755] Lustre: Skipped 1 previous similar message [ 3324.407205] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3324.410119] Lustre: Skipped 2 previous similar messages [ 3328.244770] Lustre: server umount lustre-MDT0000 complete [ 3330.020901] LustreError: 126802:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3330.032032] LustreError: 126802:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3331.042344] LustreError: 127300:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766415712 with bad export cookie 16137264387819364504 [ 3331.051814] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3331.301838] Lustre: server umount lustre-MDT0001 complete [ 3339.086485] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 3341.037470] Key type lgssc unregistered [ 3341.252650] LNet: 128033:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3341.257539] LNetError: 128033:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3341.269412] LNet: Removed LNI 192.168.204.145@tcp [ 3341.902803] Key type .llcrypt unregistered [ 3341.905533] Key type ._llcrypt unregistered [ 3357.868516] Key type ._llcrypt registered [ 3357.871709] Key type .llcrypt registered [ 3357.959950] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 3358.607693] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3358.624921] alg: No test for adler32 (adler32-zlib) [ 3359.607207] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 3359.787952] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 3361.407115] Key type lgssc registered [ 3362.428021] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3369.174223] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3369.211572] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3370.553664] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3373.583724] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 3379.685209] LustreError: 129407:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3379.694832] LustreError: 129407:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3381.448850] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3381.934767] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3385.278055] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 3389.834909] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3392.878934] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3399.099200] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3399.415931] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3404.452391] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 3409.178052] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3415.806245] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3428.240441] Lustre: *** cfs_fail_loc=159, val=0*** [ 3444.767454] Lustre: lustre-MDT0000: Client 0713b0b0-cf48-4c32-970f-b99bf1e3e720 (at 192.168.204.45@tcp) reconnecting [ 3447.117187] Lustre: *** cfs_fail_loc=159, val=0*** [ 3463.727457] Lustre: lustre-MDT0001: Client 0713b0b0-cf48-4c32-970f-b99bf1e3e720 (at 192.168.204.45@tcp) reconnecting [ 3464.882670] Lustre: *** cfs_fail_loc=159, val=0*** [ 3464.884474] Lustre: Skipped 3 previous similar messages [ 3480.081831] Lustre: lustre-MDT0001: Client 0713b0b0-cf48-4c32-970f-b99bf1e3e720 (at 192.168.204.45@tcp) reconnecting [ 3483.281345] Lustre: 132130:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3485.573923] Lustre: *** cfs_fail_loc=159, val=0*** [ 3485.587750] Lustre: Skipped 4 previous similar messages [ 3501.100727] Lustre: lustre-MDT0000: Client 846e519f-b8cd-41d7-9b66-3dd0e4225380 (at 192.168.204.45@tcp) reconnecting [ 3502.418163] Lustre: *** cfs_fail_loc=159, val=0*** [ 3502.421346] Lustre: Skipped 14 previous similar messages [ 3518.960427] Lustre: lustre-MDT0000: Client 846e519f-b8cd-41d7-9b66-3dd0e4225380 (at 192.168.204.45@tcp) reconnecting [ 3520.163578] Lustre: 132422:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3538.917655] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3538.917881] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3538.918859] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3538.932702] Lustre: Skipped 1 previous similar message [ 3543.675231] LustreError: 132546:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3543.829133] Lustre: server umount lustre-OST0000 complete [ 3549.153221] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3549.160535] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3549.168285] Lustre: Skipped 2 previous similar messages [ 3553.066547] LustreError: 132748:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3553.070439] LustreError: 132748:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3553.125985] Lustre: server umount lustre-MDT0000 complete [ 3554.276689] LustreError: 129402:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3554.290795] LustreError: 129402:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3556.026815] LustreError: 129388:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766415937 with bad export cookie 606845306478912043 [ 3556.030954] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3556.039376] LustreError: 129388:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3556.161365] LustreError: 132950:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3556.163371] LustreError: 132950:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3556.339833] Lustre: server umount lustre-MDT0001 complete [ 3564.697857] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 3566.855812] Key type lgssc unregistered [ 3567.112274] LNet: 133480:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3567.126154] LNetError: 133480:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3568.169735] LNet: Removed LNI 192.168.204.145@tcp [ 3568.811679] Key type .llcrypt unregistered [ 3568.814180] Key type ._llcrypt unregistered [ 3587.256600] Key type ._llcrypt registered [ 3587.258076] Key type .llcrypt registered [ 3587.374451] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 3588.113580] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3588.157724] alg: No test for adler32 (adler32-zlib) [ 3589.113860] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 3589.275902] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 3590.911218] Key type lgssc registered [ 3591.694759] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3597.691313] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3597.719134] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3599.121649] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3602.051950] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 3608.564223] LustreError: 134855:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3608.580205] LustreError: 134855:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3608.716544] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3609.024962] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3612.396494] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 3616.884638] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3619.912787] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3625.825920] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3626.075080] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3629.926863] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 3633.707270] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3634.288381] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3644.532556] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3649.145571] Lustre: 137124:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3650.942581] Lustre: 137172:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3652.077262] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3652.684568] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3654.641273] Lustre: 137318:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3655.210254] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3655.761139] Lustre: 137416:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3660.258516] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3660.265797] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3660.272562] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3664.024140] LustreError: 137514:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3664.219867] Lustre: server umount lustre-OST0000 complete [ 3670.499207] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3670.502397] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3670.516032] Lustre: Skipped 1 previous similar message [ 3670.519974] LustreError: Skipped 1 previous similar message [ 3670.527191] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3670.532741] Lustre: Skipped 2 previous similar messages [ 3673.174165] LustreError: 137723:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3673.176788] LustreError: 137723:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3673.251912] Lustre: server umount lustre-MDT0000 complete [ 3675.616644] LustreError: 135919:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3675.627678] LustreError: 135919:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3675.973953] LustreError: 134834:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766416057 with bad export cookie 5552517301506617632 [ 3675.979925] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3675.999072] LustreError: 134834:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3676.125809] LustreError: 137926:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3676.145972] LustreError: 137926:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3676.336700] Lustre: server umount lustre-MDT0001 complete [ 3683.379381] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 3685.158391] Key type lgssc unregistered [ 3685.378599] LNet: 138458:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3685.382443] LNetError: 138458:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3685.393709] LNet: Removed LNI 192.168.204.145@tcp [ 3685.897813] Key type .llcrypt unregistered [ 3685.900870] Key type ._llcrypt unregistered [ 3701.915694] Key type ._llcrypt registered [ 3701.918100] Key type .llcrypt registered [ 3702.022558] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 3702.543227] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3702.561942] alg: No test for adler32 (adler32-zlib) [ 3703.557844] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 3703.727755] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 3705.375380] Key type lgssc registered [ 3706.207423] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3711.984107] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3712.000549] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3713.311574] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3715.704441] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 3721.684937] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3722.067666] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3724.831841] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 3728.329663] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3730.388219] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3734.405826] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3734.635692] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3737.714286] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 3738.992891] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 3742.618285] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3746.919692] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 3750.544187] Lustre: *** cfs_fail_loc=159, val=0*** [ 3766.766902] Lustre: lustre-MDT0000: Client 04e8dd89-348f-4bfa-b1fd-c99828b8cab7 (at 192.168.204.45@tcp) reconnecting [ 3783.144311] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3783.147348] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3783.164687] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3783.167000] Lustre: Skipped 1 previous similar message [ 3788.261989] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3788.268590] Lustre: Skipped 1 previous similar message [ 3788.753478] LustreError: 142212:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3788.862501] Lustre: server umount lustre-OST0000 complete [ 3793.379269] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3793.386882] Lustre: Skipped 1 previous similar message [ 3793.389913] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3793.392612] Lustre: Skipped 1 previous similar message [ 3797.897466] LustreError: 142414:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3797.911491] LustreError: 142414:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3797.983891] Lustre: server umount lustre-MDT0000 complete [ 3798.497451] LustreError: 140562:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3798.514534] LustreError: 140562:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3800.970958] LustreError: 140556:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766416182 with bad export cookie 6246012435213382246 [ 3800.980152] LustreError: 140556:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3800.981493] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3801.117555] LustreError: 142616:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3801.129228] LustreError: 142616:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3801.301360] Lustre: server umount lustre-MDT0001 complete [ 3808.722674] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 3811.147569] Key type lgssc unregistered [ 3811.453671] LNet: 143148:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3811.458308] LNetError: 143148:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3811.469271] LNet: Removed LNI 192.168.204.145@tcp [ 3812.190187] Key type .llcrypt unregistered [ 3812.192747] Key type ._llcrypt unregistered [ 3830.124553] Key type ._llcrypt registered [ 3830.126455] Key type .llcrypt registered [ 3830.201036] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 3830.864644] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3830.926931] alg: No test for adler32 (adler32-zlib) [ 3831.894821] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 3832.062714] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 3833.719350] Key type lgssc registered [ 3834.609743] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3840.040712] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3840.058482] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3841.446179] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3844.645389] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 3850.723656] LustreError: 144522:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3850.754105] LustreError: 144522:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3853.033706] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3853.400508] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3856.524968] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 3860.900579] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3863.634404] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3869.260675] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3869.609323] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3874.365669] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 3877.828720] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 3879.521937] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3886.015735] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 3906.822745] Lustre: lustre-OST0000: evicting a129c746-86fc-4f48-8c60-7d97e6cefdb4 (at 192.168.204.45@tcp) by administrative request [ 3906.835491] Lustre: Skipped 1 previous similar message [ 3934.179214] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3934.184055] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3934.194275] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3937.250640] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3937.257141] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3937.257488] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3940.031297] LustreError: 147329:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3940.113958] Lustre: server umount lustre-OST0000 complete [ 3945.441735] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3945.445223] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3945.451821] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3945.454617] Lustre: Skipped 1 previous similar message [ 3947.492630] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3947.498853] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3948.711706] LustreError: 147530:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3948.722488] LustreError: 147530:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3948.874041] Lustre: server umount lustre-MDT0000 complete [ 3951.586414] LustreError: 144503:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766416333 with bad export cookie 651141000742333056 [ 3951.590946] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3951.593799] LustreError: 144503:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3951.707634] LustreError: 147732:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3951.710389] LustreError: 147732:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3951.836284] Lustre: server umount lustre-MDT0001 complete [ 3959.554765] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 3961.381206] Key type lgssc unregistered [ 3961.599651] LNet: 148264:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3961.603343] LNetError: 148264:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3961.612714] LNet: Removed LNI 192.168.204.145@tcp [ 3962.112566] Key type .llcrypt unregistered [ 3962.114244] Key type ._llcrypt unregistered [ 4016.915940] Key type ._llcrypt registered [ 4016.917204] Key type .llcrypt registered [ 4016.995635] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 4017.902666] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4017.921572] alg: No test for adler32 (adler32-zlib) [ 4018.901972] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 4019.110044] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 4020.743176] Key type lgssc registered [ 4021.608879] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4027.704644] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4027.731332] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4029.077394] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4032.266700] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 4038.529771] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4038.626700] LustreError: 150712:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4038.639116] LustreError: 150712:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4038.835842] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4041.581788] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 4045.448833] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4047.650766] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4052.243755] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4052.525318] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4053.559444] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4056.877755] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 4061.129102] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4063.807344] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4073.953556] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4073.956273] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4073.959323] LustreError: Skipped 1 previous similar message [ 4073.967339] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4073.974073] Lustre: Skipped 1 previous similar message [ 4079.074599] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4079.087173] Lustre: Skipped 1 previous similar message [ 4084.193903] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4084.198786] Lustre: Skipped 1 previous similar message [ 4085.215134] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4085.246174] LustreError: 152964:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4085.332264] Lustre: server umount lustre-OST0000 complete [ 4089.312924] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4089.315870] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4089.317640] Lustre: Skipped 2 previous similar messages [ 4089.324017] Lustre: Skipped 1 previous similar message [ 4093.727771] LustreError: 153168:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4093.732137] LustreError: 153168:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4093.808955] Lustre: server umount lustre-MDT0000 complete [ 4094.432853] LustreError: 151829:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4094.449303] LustreError: 151829:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4096.291912] LustreError: 153169:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766416477 with bad export cookie 4835130901647024063 [ 4096.294513] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4096.297508] LustreError: 153169:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4096.396887] LustreError: 153370:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4096.402204] LustreError: 153370:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4096.527653] Lustre: server umount lustre-MDT0001 complete [ 4103.615936] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 4105.491724] Key type lgssc unregistered [ 4105.708337] LNet: 153901:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4105.711971] LNetError: 153901:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4105.722855] LNet: Removed LNI 192.168.204.145@tcp [ 4106.314697] Key type .llcrypt unregistered [ 4106.321697] Key type ._llcrypt unregistered [ 4122.033920] Key type ._llcrypt registered [ 4122.035718] Key type .llcrypt registered [ 4122.106391] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 4122.848146] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4122.867417] alg: No test for adler32 (adler32-zlib) [ 4123.870854] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 4124.048780] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 4125.687170] Key type lgssc registered [ 4126.486701] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4131.523584] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4136.997350] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 10:15:18 (1766416518) [ 4148.405643] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_hostid [ 4156.200454] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 4162.964484] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4167.708466] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4172.012511] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4175.948169] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4181.069795] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4181.111925] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4181.127489] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4182.296669] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4182.340561] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4182.402421] Lustre: lustre-MDT0000: new disk, initializing [ 4182.491791] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4182.503686] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4185.410739] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 4192.844394] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4192.905483] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4192.975809] Lustre: 158100:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4193.030632] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4193.033313] Lustre: Skipped 1 previous similar message [ 4193.089669] Lustre: lustre-MDT0001: new disk, initializing [ 4193.182864] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4193.234374] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4193.244970] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4196.123798] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 4202.127585] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4204.871395] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4210.102911] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4210.159461] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4210.338114] Lustre: lustre-OST0000: new disk, initializing [ 4210.341158] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4210.423156] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4214.871806] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 4218.414946] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4218.434062] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4218.534199] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4220.672525] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4229.090529] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4229.097696] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4229.101575] Lustre: Skipped 1 previous similar message [ 4229.108896] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4233.396131] LustreError: 160687:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4233.507210] Lustre: server umount lustre-OST0000 complete [ 4239.329262] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4239.334429] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4239.335070] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4239.335073] Lustre: Skipped 1 previous similar message [ 4242.015576] LustreError: 160888:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 4242.020079] LustreError: 160888:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4242.076603] Lustre: server umount lustre-MDT0000 complete [ 4244.403744] LustreError: 158091:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766416625 with bad export cookie 12739210568182197389 [ 4244.407396] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4244.411217] LustreError: 158091:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4244.518136] LustreError: 161090:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4244.522833] LustreError: 161090:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4244.636397] Lustre: server umount lustre-MDT0001 complete [ 4250.729837] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 4252.513122] Key type lgssc unregistered [ 4252.693908] LNet: 161622:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4252.703324] LNetError: 161622:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4252.719583] LNet: Removed LNI 192.168.204.145@tcp [ 4253.285914] Key type .llcrypt unregistered [ 4253.288347] Key type ._llcrypt unregistered [ 4270.118729] Key type ._llcrypt registered [ 4270.121522] Key type .llcrypt registered [ 4270.235854] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 4270.796614] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4270.860554] alg: No test for adler32 (adler32-zlib) [ 4271.819603] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 4271.973994] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 4273.599192] Key type lgssc registered [ 4274.463739] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4279.802902] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4279.819073] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4281.179096] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4283.976531] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 4289.488701] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4289.793544] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4292.110676] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 4295.249872] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4297.631800] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4301.471257] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4301.666896] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4305.357547] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 4305.782184] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4309.100420] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4318.369752] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4318.383570] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4318.398789] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4318.405674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4318.419115] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4319.151923] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4319.174254] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4319.188247] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4319.194166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4319.202474] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4319.885568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4320.569857] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4320.576652] Lustre: Skipped 1 previous similar message [ 4320.591180] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4320.601540] Lustre: Skipped 1 previous similar message [ 4320.605786] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4320.611080] Lustre: Skipped 1 previous similar message [ 4320.614609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4320.630209] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4320.639512] Lustre: Skipped 1 previous similar message [ 4321.246910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4321.927235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4322.621993] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4322.633138] Lustre: Skipped 2 previous similar messages [ 4322.651481] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4322.657933] Lustre: Skipped 2 previous similar messages [ 4322.662117] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4322.665118] Lustre: Skipped 2 previous similar messages [ 4322.670541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4322.677815] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4322.680856] Lustre: Skipped 2 previous similar messages [ 4323.377571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4324.130184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4324.835261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4325.581871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4326.342172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4326.976527] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4326.980378] Lustre: Skipped 5 previous similar messages [ 4326.991497] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4326.998790] Lustre: Skipped 5 previous similar messages [ 4327.002316] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4327.005554] Lustre: Skipped 5 previous similar messages [ 4327.011174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4327.024061] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4327.026918] Lustre: Skipped 5 previous similar messages [ 4327.660180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4328.314895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4328.978926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4329.590592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4330.199964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4330.866973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4331.530456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4332.204345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4332.826418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4333.537484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4334.153788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4334.871833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4335.528423] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4335.535505] Lustre: Skipped 12 previous similar messages [ 4335.558620] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4335.566691] Lustre: Skipped 12 previous similar messages [ 4335.571852] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4335.575074] Lustre: Skipped 12 previous similar messages [ 4335.577736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4335.587791] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4335.590490] Lustre: Skipped 12 previous similar messages [ 4336.245059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4336.843955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4337.453718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4338.165642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4338.950422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4339.629303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4340.251912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4340.941672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4341.598915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4342.254712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4342.904032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4343.576246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4344.261415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4344.918674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4345.638405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4346.357565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4347.009495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4347.595773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4348.182314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4348.914241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4349.599142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4350.350626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4351.041934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4351.825584] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4351.831312] Lustre: Skipped 23 previous similar messages [ 4351.841716] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4351.847153] Lustre: Skipped 23 previous similar messages [ 4351.849386] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4351.852223] Lustre: Skipped 23 previous similar messages [ 4351.854441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4351.879845] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4351.882856] Lustre: Skipped 23 previous similar messages [ 4352.704862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4353.470145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4354.214446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4354.977857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4355.650810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4356.411153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4357.116336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4357.863903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4358.538450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4359.065966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4359.670026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4360.206325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4360.780304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4361.343737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4361.971135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4362.692983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4363.335607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4364.001856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4364.685200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4365.321321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4366.043491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4366.717087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4367.349666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4367.955401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4368.665467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4369.424565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4370.124718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4370.791400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4371.466644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4372.142449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4372.797315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4373.360721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4374.035741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4374.718074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4375.414218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4376.102787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4376.775632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4377.468846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4378.085533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4378.722615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4379.482982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4380.184562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4380.889404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4381.626621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4382.296264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4382.984718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4383.677874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4384.392808] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4384.396550] Lustre: Skipped 47 previous similar messages [ 4384.411331] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4384.425509] Lustre: Skipped 47 previous similar messages [ 4384.430749] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4384.436389] Lustre: Skipped 47 previous similar messages [ 4384.439701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4384.448243] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4384.453435] Lustre: Skipped 47 previous similar messages [ 4385.161802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4385.926823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4386.609337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4387.218439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4387.831465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4388.382725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4388.989165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4389.576407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4390.260327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4390.901552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4391.539897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4392.205957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4392.859438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4393.581882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4394.203564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4394.871022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4395.534337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4396.179590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4396.868844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4397.644816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4398.282811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4398.944927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4399.706881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4400.448654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4401.163702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4402.025947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4402.787401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4403.456155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4404.204785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4404.928748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4405.712422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4406.360352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4407.055405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4407.736878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4408.425534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4409.016621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4409.667238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4410.416284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4411.099537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4411.889728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4412.697474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4413.456206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4414.198356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4414.987594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4415.795368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4416.577055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4417.281165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4418.007919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4418.753330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4419.505600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4420.227104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4420.999077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4421.690521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4422.354710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4423.095443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4423.816695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4424.562848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4425.363673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4426.222444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4427.013656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4427.735414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4428.387105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4429.219210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4430.099271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4430.922936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4431.656109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4432.422879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4433.163777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4433.832826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4434.505241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4435.222850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4435.847328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.579575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4437.410561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4438.201179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4438.997994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4439.646079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4440.357367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4441.055087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4441.795814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4442.479505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4443.062875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4443.754193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4444.401979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4445.033858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4445.603295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4446.156844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4446.681382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4447.272501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4447.863663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4448.416163] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4448.422475] Lustre: Skipped 90 previous similar messages [ 4448.434689] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4448.443035] Lustre: Skipped 90 previous similar messages [ 4448.446781] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4448.450385] Lustre: Skipped 90 previous similar messages [ 4448.453688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4448.460253] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4448.469906] Lustre: Skipped 90 previous similar messages [ 4449.052735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4449.702985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4450.291048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4450.875796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4451.401883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4451.975701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4452.616546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4453.243282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4453.996631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4454.645095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4455.277337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4455.959965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4456.725738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4457.413446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4458.153864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4458.795579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4459.564271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4460.277220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4460.913155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4461.533792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4462.235663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4462.853617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4463.532216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4464.177859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4464.875704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4465.511494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4466.277171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4467.179647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4467.865726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4468.579282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4469.302631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4470.021458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4470.824966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4471.516721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4472.238560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4472.837838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4473.497172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4474.173168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4474.882317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4475.610038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4476.314708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4477.002691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4477.657140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4478.284775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4478.967137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4479.588414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4480.173203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4480.747204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4481.328607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4481.900501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4482.466751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4483.143452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4483.803824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4484.431567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4484.991685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4485.522247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4486.137373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4486.717552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4487.321168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4487.833710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4488.380713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4489.058284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4489.626080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4490.157439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4490.649592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4491.255561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4491.915192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4492.473776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4493.075962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4493.742253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4494.419438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4495.053439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4495.673984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4496.312317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4496.903916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4497.436102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4498.027371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4498.678370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4499.251071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4499.840699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4500.483231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4501.042809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4501.675555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4502.285467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4502.884075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4503.570874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4504.223923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4504.833702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4505.423928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4505.977807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4506.546328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4507.139304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4507.746579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4508.323389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4508.928447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4509.480489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4510.078809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4510.618656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4511.209879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4511.761268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4512.301672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4512.872975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4513.450712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4513.973388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4514.583319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4515.185290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4515.733756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4516.262198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4516.811571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4517.392440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4517.979677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4518.673450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4519.422589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4520.044718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4520.624898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4521.185784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4521.687154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4522.196692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4522.681341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4523.236984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4523.734480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4524.230218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4524.770898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4525.325222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4525.927982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4526.463430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4526.974296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4527.503515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4528.018910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4528.548850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4529.104638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4529.661306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4530.235581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4530.782654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4531.313905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4531.821557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4532.339235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4532.892197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4533.430670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4533.976141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4534.536161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4535.143197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4535.770364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4536.413819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4536.968408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4537.463907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4538.058174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4538.563926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4539.120135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4539.666892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4540.257759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4540.892278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4541.379698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4541.864161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4542.407910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4542.897502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4543.387674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4543.876751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4544.365597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4544.800328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4545.255128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4545.734820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4546.172421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4546.620375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4547.027829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4547.456417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4547.899965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4548.378468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4548.907507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4549.452554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4549.976302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4550.583295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4551.195919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4551.725519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4552.350436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4552.897146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4553.415522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4553.990827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4554.510979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4555.048736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4555.677949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4556.161892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4556.607633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4557.034893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4557.464168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4557.948672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4558.416121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4558.891309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4559.331798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4559.711286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4560.112320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4560.563604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4561.055190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4561.571594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4562.047436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4562.511329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4562.986466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4563.461391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4563.933529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4564.464362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4564.991820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4565.610661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4566.172348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4566.757597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4567.256209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4567.774132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4568.265459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4568.737862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4569.240953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4569.703794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4570.135574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4570.604564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4571.070294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4571.526553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4571.978906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4572.401932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4572.790977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4573.174790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4573.569567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4573.970279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4574.424715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4574.872438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4575.332966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4575.832443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4576.335068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4576.820643] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4576.823277] Lustre: Skipped 225 previous similar messages [ 4576.839154] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4576.846249] Lustre: Skipped 225 previous similar messages [ 4576.849088] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4576.852839] Lustre: Skipped 225 previous similar messages [ 4576.856605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4576.866490] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4576.869598] Lustre: Skipped 225 previous similar messages [ 4577.384098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4577.860622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4578.323811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4578.795084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4579.278458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4579.756149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4580.245806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4580.803975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4581.318847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4581.844671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4582.393171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4582.840973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4583.298775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4583.769994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4584.257175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4584.718342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4585.168386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4585.620425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4586.102875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4586.517815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4586.940460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4587.365556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4587.804965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4588.237358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4588.728220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4589.189867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4589.606795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4590.050978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4590.496338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4590.940748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4591.357298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4591.787884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4592.271049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4592.786652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4593.324112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4593.916692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4594.493100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4595.064596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4595.621303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4596.197747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4596.695834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4597.224156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4597.712033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4598.204415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4598.751409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4599.393147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4599.941493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4600.448498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4601.057055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4601.603511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4602.145418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4602.697239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4603.224069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4603.719471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4604.189600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4604.660890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4605.125501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4605.602491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4606.108381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4606.622287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4607.130940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4607.673711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4608.219193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4608.722919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4609.182353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4609.705771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4610.290276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4610.861774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4611.411259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4611.958993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4612.484215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4612.973827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4613.448986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4613.936533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4614.393106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4614.857345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4615.331087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4615.810338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4616.408844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4617.026324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4617.562519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4618.095055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4618.664876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4619.174516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4619.616731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4620.208520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4620.663021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4621.164507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4621.601592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4622.058935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4622.494586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4622.944568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4623.397286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4623.817517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4624.256680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4624.689199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4625.159727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4625.701505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4626.174778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4626.598587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4627.042923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4627.477945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4627.897556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4628.342662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4628.780308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4629.189101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4629.613864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4630.055587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4630.475840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4630.915481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4631.403566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4631.864308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4632.373380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4632.910765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4633.395844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4633.948403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4634.413595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4634.912127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4635.431782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4635.941282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4636.391451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4636.930475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4637.373895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4637.814565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4638.255022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4638.713134] LustreError: 163017:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4638.721237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4638.721692] LustreError: 163017:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4639.199171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4639.687089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4640.150629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4640.678561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4641.110574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4641.544849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4641.996122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4642.450932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4642.929036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4643.377605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4643.904876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4644.379484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4644.793761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4645.213243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4645.647800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4646.073346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4646.500279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4646.931686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4647.392817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4647.846045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4648.301083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4648.791279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4649.257483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4649.750595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4650.253874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4650.693272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4651.145625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4651.635218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4652.140673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4652.776379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4653.268752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4653.765867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4654.330888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4654.845881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4655.305143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4655.799351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4656.319168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4656.788882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4657.242702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4657.757157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4658.188450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4658.688383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4659.180866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4659.609163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4660.047280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4660.501373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4660.942418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4661.372322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4661.809713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4662.256233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4662.732705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4663.175932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4663.611256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4664.100571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4664.600133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4665.028963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4665.507204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4665.968200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4666.536297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4666.990566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4667.402469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4667.838859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4668.278788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4668.697977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4669.118838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4669.534347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4669.980780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4670.412878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4670.818310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4671.240423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4671.657272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4672.081553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4672.515325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4673.010835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4673.487276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4673.931158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4674.372723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4674.843948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4675.298473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4675.727893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4676.181922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4676.643886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4677.106754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4677.527967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4678.044061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4678.497615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4679.008233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4679.482462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4679.934093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4680.375635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4680.821830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4681.281266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4681.871077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4682.403613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4682.943577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4683.456294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4683.931726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4684.425585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4684.880892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4685.350744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4685.814443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4686.272206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4686.706978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4687.161833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4687.617062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4688.090286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4688.562464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4689.002499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4689.455264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4689.938363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4690.333110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4690.749210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4691.159116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4691.592726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4692.046243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4692.492244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4692.915643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4693.379899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4693.847814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4694.288511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4694.707929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4695.147397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4695.597326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4696.040557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4696.522792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4696.986916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4697.440546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4697.875401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4698.311328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4698.809180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4699.250851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4699.716421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4700.142411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4700.561270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4700.994630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4701.423536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4701.878305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4702.267401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4702.684065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4703.109515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4703.529337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4703.966851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4704.420670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4704.873748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4705.325744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4705.754734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4706.173223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4706.575554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4706.973604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4707.385574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4707.785208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4708.220799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4708.611910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4709.002194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4709.423840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4709.812448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4710.196933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4710.603692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4710.982848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4711.359485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4711.750784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4712.125905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4712.500680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4712.913482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4713.309872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4713.704989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4714.146338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4714.519575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4714.913867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4715.340314] LustreError: 163017:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4715.345080] LustreError: 163017:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4715.346545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4715.785224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4716.201640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4716.634950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4717.050543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4717.473568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4717.889844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4718.330196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4718.743928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4719.210975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4719.655671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4720.077996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4720.480455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4720.974974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4721.421187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4721.881195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4722.333956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4722.792019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4723.308698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4723.761965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4724.262458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4724.750576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4725.205230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4725.701390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4726.175538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4726.767623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4727.300681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4727.864817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4728.417210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4729.025685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4729.648478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4730.210408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4730.722610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4731.300486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4731.905480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4732.445660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4732.939647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4733.412663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4733.863954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4734.315822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4734.768972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4735.244733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4735.687190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4736.116236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4736.569435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4736.983280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4737.397229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4737.860737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4738.312084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4738.730741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4739.218094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4739.688085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4740.125406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4740.520163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4740.952524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4741.389475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4741.823130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4742.257060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4742.774720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4743.300682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4743.836766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4744.370085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4744.866534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4745.348731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4745.830502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4746.281535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4746.776180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4747.238886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4747.699112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4748.095616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4748.547788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4748.996521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4749.430611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4749.880279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4750.336360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4750.784490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4751.230267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4751.673529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4752.096437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4752.486571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4752.917124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4753.340418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4753.803603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4754.239094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4754.694236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4755.225256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4755.729744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4756.155089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4756.581491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4757.042087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4757.519348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4757.991528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4758.482811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4758.961575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4759.380854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4759.813801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4760.229508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4760.637275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4761.040899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4761.491695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4761.953660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4762.409469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4762.832246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.296313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.765412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4764.268744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4764.797160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4765.253410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4765.720364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4766.171805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4766.721476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4767.170282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4767.619472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4768.038214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4768.526765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4769.066600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4769.538668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4770.059443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4770.535352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4770.971762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4771.452110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4771.944547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4772.398636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4772.874288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4773.339317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4773.837746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4774.294872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4774.780361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4775.200655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4775.588705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4775.991178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4776.416446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4776.835510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4777.243804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4777.662403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4778.091810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4778.513130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4778.985364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.450308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.900401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4780.365354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4780.828963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4781.250664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4781.687317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4782.173473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4782.638810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4783.109381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4783.535956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4783.944566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.375830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.795269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4785.199826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4785.596825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4786.013400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4786.466145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4786.961202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4787.386984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4787.781145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4788.176853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4788.622392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4789.063960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4789.495116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4789.925430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4790.330845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4790.766733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4791.220458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4791.645500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4792.092035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4792.509736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4792.943138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4793.416678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4793.844077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.287122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.789612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4795.259114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4795.699595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.235624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.749226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4797.251752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4797.707302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.167882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.644068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.164890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.702598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4800.134200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4800.565940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4801.023817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4801.492127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4801.933230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.373351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.815674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.235333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.646736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.096885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.585948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.016303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.413473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.828435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4806.268571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4806.666068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.109538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.553119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.962899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.381409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.780625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4809.157240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4809.537164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4809.939463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.353150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.752835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.166693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.544264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.928496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.346523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.728791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.121855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.534250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.935920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.352367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.753472] LustreError: 163017:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4814.758741] LustreError: 163017:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4814.760093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.181506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.596970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.009922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.413045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.798505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.229080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.649576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.054664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.461559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.873648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.284308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.694515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.156605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.597241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.048378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.468719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.906916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.395401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.903991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4823.393366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4823.854176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.266627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.692689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.125085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.581568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.015861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.464064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.940852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4827.375828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4827.797426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.318369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.735500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.173289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.594821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.037089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.501391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.930274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.370154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.839023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.244471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.705831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4833.133071] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4833.135757] Lustre: Skipped 556 previous similar messages [ 4833.143942] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4833.150123] Lustre: Skipped 556 previous similar messages [ 4833.153473] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4833.157022] Lustre: Skipped 556 previous similar messages [ 4833.159746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4833.165734] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4833.169450] Lustre: Skipped 556 previous similar messages [ 4833.585192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.008503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.442721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.858310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.302918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.741360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.150622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.520821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.932821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4837.314670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4837.697801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.109270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.492156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.873216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.250666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.629839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.010100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.416862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.842535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4841.254187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4841.674721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.110730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.525518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.937462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4843.354936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4843.741983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.179922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.574281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.926963] LustreError: 165824:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 5084.931554] LustreError: 163017:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 5271.016923] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 5271.019972] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5271.020468] LustreError: Skipped 6 previous similar messages [ 5272.544559] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5272.548918] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5272.553571] LustreError: Skipped 1 previous similar message [ 5274.750181] LustreError: 216796:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5274.848893] Lustre: server umount lustre-OST0000 complete [ 5277.665616] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5277.669237] Lustre: Skipped 2 previous similar messages [ 5282.766684] LustreError: 216999:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 5282.769088] LustreError: 216999:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5282.784990] LustreError: 162997:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5282.793408] LustreError: 162997:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5282.866424] Lustre: server umount lustre-MDT0000 complete [ 5284.547463] LustreError: 162977:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766417666 with bad export cookie 12836909507572055055 [ 5284.550507] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5284.553172] LustreError: 162977:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5284.624740] LustreError: 217200:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5284.627783] LustreError: 217200:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5284.709544] Lustre: server umount lustre-MDT0001 complete [ 5289.185143] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 5290.482137] Key type lgssc unregistered [ 5290.643749] LNet: 217732:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5290.647832] LNetError: 217732:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5290.656566] LNet: Removed LNI 192.168.204.145@tcp [ 5291.071509] Key type .llcrypt unregistered [ 5291.073229] Key type ._llcrypt unregistered [ 5302.271742] Key type ._llcrypt registered [ 5302.273209] Key type .llcrypt registered [ 5302.324303] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 5302.812362] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5302.826375] alg: No test for adler32 (adler32-zlib) [ 5303.728773] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 5303.855840] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 5305.455489] Key type lgssc registered [ 5306.013868] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5309.596663] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5309.607824] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5310.835781] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5312.427492] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5316.237566] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5316.419672] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5318.321447] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5321.188633] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5323.209781] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5326.610544] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5326.801366] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5329.978622] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5332.461068] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5336.045130] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 5344.997963] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5345.880213] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5345.888132] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5345.892675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.899194] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5345.900085] LustreError: 219126:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 5349.372613] Lustre: lustre-OST0000: Not available for connect from 192.168.204.45@tcp (stopping) [ 5351.396772] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5351.398523] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5351.398880] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5351.403157] Lustre: Skipped 1 previous similar message [ 5354.477783] Lustre: lustre-OST0000: Not available for connect from 192.168.204.45@tcp (stopping) [ 5354.481963] Lustre: Skipped 1 previous similar message [ 5356.516873] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5356.519851] Lustre: Skipped 1 previous similar message [ 5361.119157] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5361.151130] LustreError: 221529:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5361.206986] Lustre: server umount lustre-OST0000 complete [ 5365.991190] LustreError: 219126:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 5389.810507] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5389.945491] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5392.215197] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5394.428051] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5407.332057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.336985] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.337536] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5407.348596] Lustre: Skipped 1 previous similar message [ 5415.891245] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 20 sec [ 5417.782825] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9a2d5133c800.ost_server_uuid 50 [ 5418.518247] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9a2d5133c800.ost_server_uuid in FULL state after 0 sec [ 5423.076117] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5423.077410] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5423.088458] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5423.092409] Lustre: Skipped 2 previous similar messages [ 5426.101251] LustreError: 222872:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5426.107681] LustreError: 222872:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5426.167808] Lustre: server umount lustre-OST0000 complete [ 5428.193486] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5428.201699] Lustre: Skipped 1 previous similar message [ 5433.312231] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5433.315715] Lustre: Skipped 3 previous similar messages [ 5433.984152] LustreError: 223073:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 5433.986108] LustreError: 223073:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5434.018711] Lustre: server umount lustre-MDT0000 complete [ 5435.422224] LustreError: 219087:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766417816 with bad export cookie 11039352336131594057 [ 5435.425266] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5435.428333] LustreError: 219087:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5435.563349] Lustre: server umount lustre-MDT0001 complete [ 5439.578148] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 5440.720330] Key type lgssc unregistered [ 5440.874643] LNet: 223804:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5440.878660] LNetError: 223804:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5440.891390] LNet: Removed LNI 192.168.204.145@tcp [ 5441.248928] Key type .llcrypt unregistered [ 5441.250592] Key type ._llcrypt unregistered [ 5455.242813] Key type ._llcrypt registered [ 5455.244195] Key type .llcrypt registered [ 5455.308231] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 5455.709787] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5455.755418] alg: No test for adler32 (adler32-zlib) [ 5456.655794] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 5456.762389] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 5458.375282] Key type lgssc registered [ 5458.874740] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5463.131235] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5463.141558] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5464.376941] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5465.869832] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5469.681657] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5469.849513] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5471.415696] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5472.620287] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5475.787783] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5475.963671] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5478.759856] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5483.059784] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5483.095490] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5483.177896] Lustre: lustre-OST0001: new disk, initializing [ 5483.180730] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5483.231714] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5485.729023] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5485.733745] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5485.782553] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5486.050894] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5488.813540] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 5492.305289] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5494.335403] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5509.602852] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5509.607163] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5509.607481] Lustre: Skipped 2 previous similar messages [ 5509.610471] Lustre: Skipped 2 previous similar messages [ 5514.722118] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5514.724866] Lustre: Skipped 4 previous similar messages [ 5515.031492] LustreError: 228852:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 5515.085212] Lustre: server umount lustre-MDT0000 complete [ 5516.933537] LustreError: 228562:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766417898 with bad export cookie 12475229162533655275 [ 5516.937186] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5516.937949] LustreError: 228562:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5516.943418] LustreError: 226212:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 5517.018462] LustreError: 229055:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 5517.020592] LustreError: 229055:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5517.100179] Lustre: server umount lustre-MDT0001 complete [ 5529.083238] LustreError: 229255:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5529.085314] LustreError: 229255:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5529.136171] Lustre: server umount lustre-OST0000 complete [ 5541.272456] LustreError: 229457:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5541.277161] LustreError: 229457:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5541.372272] Lustre: server umount lustre-OST0001 complete [ 5542.768330] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5542.805466] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5543.268244] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5543.303457] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5543.751505] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5543.782333] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5544.241717] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5544.271939] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5550.099342] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 5554.369951] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5554.598365] LustreError: 230864:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5554.608908] LustreError: 230864:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5554.644259] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5556.288711] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5559.777371] LustreError: 230864:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5560.166776] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5562.097887] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5563.661639] Lustre: 231975:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5566.890534] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5567.063208] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5567.065752] Lustre: Skipped 1 previous similar message [ 5569.730695] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5570.155052] LustreError: 232331:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5570.164201] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 5573.934127] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5575.990413] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 5577.097963] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5581.101643] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49761) [ 5586.638955] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5588.400394] Lustre: 233825:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5600.975725] LustreError: 234134:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5600.978741] LustreError: 234134:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5601.019871] Lustre: server umount mylustre-MDT0000 complete [ 5601.248148] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5601.254124] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5601.270179] LustreError: 230864:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5602.802896] LustreError: 230839:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766417984 with bad export cookie 12475229162533658124 [ 5602.807145] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5602.808019] LustreError: 230839:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5602.952237] Lustre: server umount mylustre-MDT0001 complete [ 5615.093102] LustreError: 234536:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5615.095624] LustreError: 234536:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 5615.120862] Lustre: server umount mylustre-OST0000 complete [ 5626.875662] Lustre: server umount mylustre-OST0001 complete [ 5628.229380] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5628.268132] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5628.751167] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5628.804480] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5629.304347] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5629.339702] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5629.860544] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5629.905342] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5636.476929] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 5640.924933] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5641.135614] LustreError: 236145:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5641.144537] LustreError: 236145:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5641.169906] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5641.172511] Lustre: Skipped 1 previous similar message [ 5642.772612] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5646.676020] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5648.533892] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5649.692855] Lustre: 237253:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5652.350268] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5654.813408] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5656.613425] LustreError: 238182:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5656.618595] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 5656.621219] LustreError: 238182:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5658.325493] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5659.699837] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:65) [ 5660.731636] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5665.832945] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49793) [ 5669.488960] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5670.921391] Lustre: 239103:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5681.121898] Lustre: tfs-MDT0000-lwp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5681.123925] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 5681.128746] Lustre: Skipped 6 previous similar messages [ 5681.133174] Lustre: Skipped 3 previous similar messages [ 5684.161617] LustreError: 239354:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5684.164340] LustreError: 239354:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 5684.204248] Lustre: server umount tfs-MDT0000 complete [ 5685.618252] LustreError: 236119:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766418067 with bad export cookie 12475229162533660301 [ 5685.622069] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5685.622548] LustreError: 236119:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5685.753976] Lustre: server umount tfs-MDT0001 complete [ 5697.481906] Lustre: server umount tfs-OST0000 complete [ 5709.227951] Lustre: server umount tfs-OST0001 complete [ 5710.568564] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5710.605751] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5711.101427] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5711.134055] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5711.577877] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5711.609518] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5712.064078] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5712.103203] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5717.989906] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 5722.310113] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5722.519919] LustreError: 241366:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5722.552898] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5722.556055] Lustre: Skipped 3 previous similar messages [ 5724.169397] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5727.821553] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5729.617452] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5730.792558] Lustre: 242475:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5733.590590] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5736.142622] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5737.897351] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 5739.671325] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5741.693124] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:97) [ 5742.097236] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5746.859113] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49825) [ 5750.889639] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5752.249825] Lustre: 244323:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5759.670503] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 10:42:22 (1766418142) [ 5772.773597] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5772.775091] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5772.779243] Lustre: Skipped 2 previous similar messages [ 5772.781297] Lustre: Skipped 1 previous similar message [ 5777.863156] LustreError: 244693:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5777.865723] LustreError: 244693:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 5777.889388] LustreError: 241365:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5777.899660] LustreError: 241365:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 5777.913404] Lustre: server umount lustre-MDT0000 complete [ 5779.476394] LustreError: 241342:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766418160 with bad export cookie 12475229162533662219 [ 5779.480155] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5779.482200] LustreError: 241342:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5783.009726] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5783.011267] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5783.013702] Lustre: Skipped 2 previous similar messages [ 5783.016159] Lustre: Skipped 2 previous similar messages [ 5785.683912] Lustre: server umount lustre-MDT0001 complete [ 5793.288863] Lustre: server umount lustre-OST0000 complete [ 5809.119909] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5809.207730] Lustre: server umount lustre-OST0001 complete [ 5811.799119] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_hostid [ 5815.334826] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 5819.280863] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5821.843517] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5824.440557] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5827.083043] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5829.954398] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5829.980205] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 5829.980205] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 5829.980205] [ 5829.993055] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 5830.094697] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5830.110138] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5830.156328] Lustre: lustre-MDT0000: new disk, initializing [ 5830.201513] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5830.204498] Lustre: Skipped 3 previous similar messages [ 5830.216535] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5831.761106] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5836.640544] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5836.668839] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5836.712742] Lustre: 247479:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5836.744302] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5836.747073] Lustre: Skipped 1 previous similar message [ 5836.789377] Lustre: lustre-MDT0001: new disk, initializing [ 5836.850821] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5836.855374] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5838.408587] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5843.058741] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5843.085296] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5843.197739] Lustre: lustre-OST0000: new disk, initializing [ 5843.201674] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5844.820749] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5844.826751] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5844.884686] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5845.488427] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5850.156613] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5850.182937] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5850.253445] Lustre: lustre-OST0001: new disk, initializing [ 5850.255796] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5851.675583] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5851.679326] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5851.726434] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5852.593814] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5857.644605] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 10:43:59 (1766418239) [ 5862.340307] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 10:44:04 (1766418244) [ 5875.681113] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5875.681145] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5875.684882] LustreError: Skipped 1 previous similar message [ 5875.688237] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5875.692660] Lustre: Skipped 1 previous similar message [ 5875.694568] Lustre: Skipped 3 previous similar messages [ 5880.183982] LustreError: 250898:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5880.187390] LustreError: 250898:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 5880.215065] Lustre: server umount lustre-OST0000 complete [ 5882.341536] LustreError: 250596:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5882.348948] LustreError: 250596:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 5892.579680] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5892.584093] Lustre: Skipped 10 previous similar messages [ 5896.159144] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5896.240103] Lustre: server umount lustre-MDT0000 complete [ 5897.784157] LustreError: 247469:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766418279 with bad export cookie 12475229162533663997 [ 5897.789461] LustreError: 247469:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5897.794707] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5902.816500] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5902.821072] Lustre: Skipped 3 previous similar messages [ 5904.002426] Lustre: server umount lustre-MDT0001 complete [ 5908.086317] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 5918.817994] Lustre: server umount lustre-OST0001 complete [ 5921.200856] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_hostid [ 5927.993720] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 5931.889938] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5934.409573] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5936.910813] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5939.455284] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5942.010960] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5942.035429] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5942.129070] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5942.145373] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5942.181511] Lustre: lustre-MDT0000: new disk, initializing [ 5942.215979] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5943.818439] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5948.624763] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5948.651278] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5948.695239] Lustre: 254360:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5948.825305] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5950.348100] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5953.763983] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5955.231125] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5957.882510] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5957.913433] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5960.000218] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5960.003890] Lustre: Skipped 1 previous similar message [ 5960.006235] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5960.052763] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5960.389029] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 5963.914187] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5970.402086] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5970.404197] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5970.405182] LustreError: Skipped 1 previous similar message [ 5970.411692] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5970.415011] Lustre: Skipped 2 previous similar messages [ 5972.578576] Lustre: server umount lustre-OST0000 complete [ 5974.496028] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5988.319187] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5988.391461] Lustre: server umount lustre-MDT0000 complete [ 5989.830929] LustreError: 255139:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766418371 with bad export cookie 12475229162533665467 [ 5989.834688] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5989.836642] LustreError: 255139:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5989.979327] Lustre: server umount lustre-MDT0001 complete [ 5993.968838] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 5995.115567] Key type lgssc unregistered [ 5995.257402] LNet: 257732:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5995.261223] LNetError: 257732:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5995.270517] LNet: Removed LNI 192.168.204.145@tcp [ 5995.618240] Key type .llcrypt unregistered [ 5995.619988] Key type ._llcrypt unregistered [ 6006.632074] Key type ._llcrypt registered [ 6006.633647] Key type .llcrypt registered [ 6006.681571] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 6007.020109] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6007.028087] alg: No test for adler32 (adler32-zlib) [ 6007.903106] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 6008.007856] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 6009.599159] Key type lgssc registered [ 6009.983632] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6012.987865] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6012.996236] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6014.192243] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6015.740242] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6019.118051] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6019.274277] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6020.828442] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6022.933809] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6024.341639] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6026.985579] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6027.146926] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6029.410978] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6031.565882] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6031.942422] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 6034.405181] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6034.405571] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6034.413684] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6038.700093] LustreError: 261449:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6038.740504] Lustre: server umount lustre-OST0000 complete [ 6044.641324] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6044.643314] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6044.645923] Lustre: Skipped 1 previous similar message [ 6046.333523] LustreError: 261654:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 6046.335478] LustreError: 261654:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6046.361340] Lustre: server umount lustre-MDT0000 complete [ 6047.741828] LustreError: 259133:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766418429 with bad export cookie 5520517091290446801 [ 6047.745309] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6047.746705] LustreError: 259133:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6047.816664] LustreError: 261864:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6047.818543] LustreError: 261864:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6047.882761] Lustre: server umount lustre-MDT0001 complete [ 6051.802978] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 6052.982380] Key type lgssc unregistered [ 6053.121391] LNet: 262395:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6053.124104] LNetError: 262395:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6053.136350] LNet: Removed LNI 192.168.204.145@tcp [ 6053.462889] Key type .llcrypt unregistered [ 6053.464500] Key type ._llcrypt unregistered [ 6060.815803] Key type ._llcrypt registered [ 6060.817159] Key type .llcrypt registered [ 6060.868982] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 6061.248125] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6061.306465] alg: No test for adler32 (adler32-zlib) [ 6062.180175] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 6062.277787] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 6063.871171] Key type lgssc registered [ 6064.319147] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6067.842423] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6067.850433] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6083.295269] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 6088.416556] LustreError: 263546:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.145@tcp: failed processing log, type 4: rc = -110 [ 6115.103176] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6117.367577] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6119.545585] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6120.398390] LustreError: 264209:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 6120.446462] Lustre: server umount lustre-OST0000 complete [ 6126.628544] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 6127.799937] Key type lgssc unregistered [ 6127.947657] LNet: 265044:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6127.951364] LNetError: 265044:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6127.961306] LNet: Removed LNI 192.168.204.145@tcp [ 6128.309617] Key type .llcrypt unregistered [ 6128.311840] Key type ._llcrypt unregistered [ 6143.877769] Key type ._llcrypt registered [ 6143.879443] Key type .llcrypt registered [ 6143.931554] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 6144.284856] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6144.325595] alg: No test for adler32 (adler32-zlib) [ 6145.210247] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 6145.324134] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 6146.919623] Key type lgssc registered [ 6147.339361] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6152.424565] loop0: detected capacity change from 0 to 204800000 [ 6152.545553] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6152.551750] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6152.705966] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6152.731228] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6153.124989] loop0: detected capacity change from 0 to 204800000 [ 6153.186930] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6153.191094] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6153.343742] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6153.367804] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6153.767167] loop0: detected capacity change from 0 to 204800000 [ 6153.837199] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6153.842508] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6153.894196] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6153.943735] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6154.323072] loop0: detected capacity change from 0 to 204800000 [ 6154.398597] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6154.500175] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6154.874973] loop0: detected capacity change from 0 to 204800000 [ 6154.889813] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6156.539060] loop0: detected capacity change from 0 to 204800000 [ 6156.549077] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6158.125987] loop0: detected capacity change from 0 to 204800000 [ 6158.138762] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6159.753381] loop0: detected capacity change from 0 to 204800000 [ 6159.770667] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6161.370759] loop0: detected capacity change from 0 to 204800000 [ 6161.411937] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6161.432353] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6161.441696] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6161.468565] Lustre: 268418:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6161.475578] Lustre: 268418:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 6161.480451] Lustre: 268418:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6161.485508] Lustre: 268418:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 6161.492409] Lustre: 268418:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6161.499916] Lustre: 268418:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 6161.506784] Lustre: 268418:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 6161.513716] Lustre: 268418:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6161.519369] Lustre: 268418:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 6161.523867] Lustre: 268418:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 6161.528979] Lustre: 268418:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 6161.536752] Lustre: 268418:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 6161.542827] Lustre: 268418:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 6161.548160] Lustre: 268418:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 6164.716565] LustreError: 268684:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6164.786266] Lustre: server umount MGS complete [ 6165.220638] loop0: detected capacity change from 0 to 204800000 [ 6165.272178] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6165.478791] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6165.500196] Lustre: lustre-MDT0000: Aborting client recovery [ 6165.883495] loop1: detected capacity change from 0 to 204800000 [ 6165.926780] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6165.954301] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6165.982633] Lustre: 268854:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6165.986732] Lustre: 268854:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 6165.991389] Lustre: 268854:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 6165.996803] Lustre: 268854:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6166.002156] Lustre: 268854:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6166.009809] Lustre: 268854:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6166.015463] Lustre: 268854:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 6166.023059] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6166.102084] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6166.131168] Lustre: lustre-MDT0001: Aborting client recovery [ 6166.523794] loop2: detected capacity change from 0 to 204800000 [ 6166.571279] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6166.621647] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6166.640860] Lustre: 268940:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6166.646143] Lustre: 268940:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6166.650836] Lustre: 268940:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6166.656344] Lustre: 268940:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6166.660740] Lustre: 268940:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6166.665388] Lustre: 268940:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6166.669638] Lustre: 268940:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6166.723167] Lustre: Found index 0 for lustre-OST0000, updating log [ 6166.773422] Lustre: lustre-OST0000: Aborting recovery [ 6167.172495] loop3: detected capacity change from 0 to 204800000 [ 6167.243410] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6167.278852] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6167.301143] Lustre: 269028:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6167.305521] Lustre: 269028:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6167.311117] Lustre: 269028:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6167.316527] Lustre: 269028:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6167.324895] Lustre: 269028:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6167.331968] Lustre: 269028:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6167.337586] Lustre: 269028:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6167.397087] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6167.400817] Lustre: Skipped 1 previous similar message [ 6167.416791] Lustre: lustre-OST0001: Aborting recovery [ 6172.655683] LustreError: 268797:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -108 [ 6174.689899] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6174.691157] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6174.704639] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 6174.710844] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 6175.714171] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 6175.714432] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 6178.268139] LustreError: 268765:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 383us: rc = -2 [ 6191.072619] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6191.073305] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6191.078281] Lustre: Skipped 3 previous similar messages [ 6191.080947] Lustre: Skipped 2 previous similar messages [ 6195.718730] LustreError: 269382:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6195.720746] LustreError: 269382:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 6195.771183] Lustre: server umount lustre-MDT0000 complete [ 6196.127620] LustreError: 268747:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766418577 with bad export cookie 10320495003580913998 [ 6196.129296] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6196.132348] LustreError: 268747:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6196.191682] LustreError: 268761:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6196.191904] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6196.196450] LustreError: 268761:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6196.200691] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6196.205209] Lustre: Skipped 2 previous similar messages [ 6201.313724] LustreError: 268762:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6201.314250] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6201.322734] LustreError: 268762:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6201.330204] Lustre: Skipped 1 previous similar message [ 6202.378989] LustreError: 269433:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 6202.383742] LustreError: 269433:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6202.488728] Lustre: server umount lustre-MDT0001 complete [ 6217.183122] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6217.203058] LustreError: 269485:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 6217.207925] LustreError: 269485:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6217.248734] Lustre: server umount lustre-OST0000 complete [ 6223.726797] LustreError: 269537:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6223.729658] LustreError: 269537:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6223.797487] Lustre: server umount lustre-OST0001 complete [ 6226.960754] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 10:50:09 (1766418609) [ 6238.708631] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_hostid [ 6242.540280] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 6246.683221] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6249.249828] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6251.798809] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6254.412551] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6257.229041] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6257.257995] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6257.364035] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6257.381420] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6257.425779] Lustre: lustre-MDT0000: new disk, initializing [ 6257.465922] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6257.474908] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6259.031713] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6263.880482] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6263.904212] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6263.945701] Lustre: 273084:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6263.975737] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6263.978767] Lustre: Skipped 1 previous similar message [ 6264.022763] Lustre: lustre-MDT0001: new disk, initializing [ 6264.067745] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6264.084030] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6264.088298] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6265.637509] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6269.062063] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6270.552468] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6273.278156] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6273.305370] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6273.424681] Lustre: lustre-OST0000: new disk, initializing [ 6273.429712] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 6273.429950] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6273.465897] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6275.809833] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6278.646766] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6278.650907] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6278.659743] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000400 [ 6279.290600] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6282.059163] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 6287.433036] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 6296.256616] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 6307.149919] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 6345.186029] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6345.187147] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6345.188642] LustreError: Skipped 1 previous similar message [ 6345.194157] Lustre: Skipped 1 previous similar message [ 6345.196402] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6345.199625] Lustre: Skipped 1 previous similar message [ 6347.371120] LustreError: 276769:0:(obd_class.h:479:obd_check_dev()) Device 20 not setup [ 6347.373888] LustreError: 276769:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6347.409027] Lustre: server umount lustre-OST0000 complete [ 6350.304575] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6350.308654] Lustre: Skipped 1 previous similar message [ 6355.424077] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6355.426090] Lustre: Skipped 3 previous similar messages [ 6363.103162] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6363.169906] Lustre: server umount lustre-MDT0000 complete [ 6364.489174] LustreError: 273076:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766418745 with bad export cookie 10320495003580916707 [ 6364.492095] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6364.494285] LustreError: 273076:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6364.560117] LustreError: 277172:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 6364.562909] LustreError: 277172:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 6364.626565] Lustre: server umount lustre-MDT0001 complete [ 6367.979592] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6369.493649] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6371.466810] Lustre: server umount MGS complete [ 6375.778731] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6375.976265] LustreError: 278736:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6375.992591] LustreError: 278748:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 6376.028829] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6377.598413] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6380.990564] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6381.025380] LustreError: 278736:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6381.035910] LustreError: 278736:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6382.735348] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6384.716313] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6386.087228] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6388.511512] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6390.670721] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6392.753238] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6426.594851] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6426.594878] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6426.598535] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6426.603603] Lustre: Skipped 1 previous similar message [ 6426.608607] Lustre: Skipped 3 previous similar messages [ 6431.601405] LustreError: 281414:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6431.603463] LustreError: 281414:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 6431.666926] Lustre: server umount lustre-OST0000 complete [ 6436.832616] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6436.838022] Lustre: Skipped 1 previous similar message [ 6439.411674] Lustre: server umount lustre-MDT0000 complete [ 6440.995949] LustreError: 278717:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766418822 with bad export cookie 10320495003580918548 [ 6440.998819] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6441.002502] LustreError: 278717:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6441.154930] Lustre: server umount lustre-MDT0001 complete [ 6445.240895] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 6446.440696] Key type lgssc unregistered [ 6446.576401] LNet: 282348:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6446.579516] LNetError: 282348:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6446.587618] LNet: Removed LNI 192.168.204.145@tcp [ 6446.938847] Key type .llcrypt unregistered [ 6446.940471] Key type ._llcrypt unregistered [ 6460.746032] Key type ._llcrypt registered [ 6460.747433] Key type .llcrypt registered [ 6460.798957] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_hostid [ 6467.171478] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 6467.594788] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6467.603742] alg: No test for adler32 (adler32-zlib) [ 6468.480812] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 6468.564330] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 6470.143144] Key type lgssc registered [ 6470.521607] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6473.797324] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6476.285297] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6478.687779] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6481.210568] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6484.088500] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6484.116503] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6484.124417] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6485.229691] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6485.248442] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6485.293123] Lustre: lustre-MDT0000: new disk, initializing [ 6485.334768] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6485.344627] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6486.901257] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6491.896724] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6491.931408] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6491.982182] Lustre: 286254:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6492.016774] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6492.019515] Lustre: Skipped 1 previous similar message [ 6492.066396] Lustre: lustre-MDT0001: new disk, initializing [ 6492.124345] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6492.141629] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6492.147083] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6493.730704] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6497.233570] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6498.566554] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6501.073844] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6501.100347] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6501.202694] Lustre: lustre-OST0000: new disk, initializing [ 6501.204746] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6501.239856] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6501.749363] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6501.753202] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6501.799563] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6503.412626] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6506.622379] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6510.074105] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 6516.802263] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 6526.882816] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 6531.405808] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 6568.416488] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6568.418033] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6568.419612] LustreError: Skipped 1 previous similar message [ 6568.423186] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6568.426174] Lustre: Skipped 1 previous similar message [ 6569.519066] LustreError: 290007:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 6569.553531] Lustre: server umount lustre-OST0000 complete [ 6573.537034] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6573.538179] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6573.540208] Lustre: Skipped 2 previous similar messages [ 6573.543429] Lustre: Skipped 1 previous similar message [ 6578.656067] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6578.658321] Lustre: Skipped 1 previous similar message [ 6583.776103] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6583.778030] Lustre: Skipped 1 previous similar message [ 6585.311092] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6585.351379] LustreError: 290210:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 6585.354404] LustreError: 290210:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6585.388752] Lustre: server umount lustre-MDT0000 complete [ 6586.630150] LustreError: 287033:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766418968 with bad export cookie 13417356429898069990 [ 6586.632658] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6586.633610] LustreError: 287033:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6586.699512] LustreError: 290413:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6586.701928] LustreError: 290413:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6586.757692] Lustre: server umount lustre-MDT0001 complete [ 6589.768190] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6591.174368] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6592.825573] LustreError: 291353:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6592.828271] LustreError: 291353:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6592.853315] Lustre: server umount MGS complete [ 6596.846197] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6597.040582] LustreError: 291988:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 6597.071193] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6598.518800] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6601.739375] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6601.888478] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6603.330750] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6605.249337] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6606.548880] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6608.966770] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6611.099956] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6612.933190] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6648.291317] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6648.291397] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6648.293202] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6648.299657] Lustre: Skipped 1 previous similar message [ 6650.286830] LustreError: 294653:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6650.289079] LustreError: 294653:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 6650.326885] Lustre: server umount lustre-OST0000 complete [ 6653.409929] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6653.410115] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6653.413022] Lustre: Skipped 1 previous similar message [ 6653.415982] LustreError: Skipped 1 previous similar message [ 6657.821524] Lustre: server umount lustre-MDT0000 complete [ 6658.528592] LustreError: 291976:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6658.535044] LustreError: 291976:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6659.283083] LustreError: 291955:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766419040 with bad export cookie 13417356429898071817 [ 6659.287954] LustreError: 291955:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6659.287975] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6659.367973] LustreError: 295057:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6659.370658] LustreError: 295057:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 6659.448289] Lustre: server umount lustre-MDT0001 complete [ 6663.273565] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 6664.477283] Key type lgssc unregistered [ 6664.618420] LNet: 295588:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6664.620965] LNetError: 295588:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6664.634452] LNet: Removed LNI 192.168.204.145@tcp [ 6664.963673] Key type .llcrypt unregistered [ 6664.965185] Key type ._llcrypt unregistered [ 6672.310396] Key type ._llcrypt registered [ 6672.311772] Key type .llcrypt registered [ 6672.351958] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_hostid [ 6678.433001] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 6678.870211] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6678.897863] alg: No test for adler32 (adler32-zlib) [ 6679.765709] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 6679.861413] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 6681.447216] Key type lgssc registered [ 6681.852255] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6685.044357] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6687.144680] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6690.235532] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 6693.075250] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6695.425541] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6695.449195] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6695.454712] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6696.547136] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6696.560289] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6696.597839] Lustre: lustre-MDT0000: new disk, initializing [ 6696.626342] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6696.633926] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6697.969885] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6702.212695] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6702.236307] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6702.327201] Lustre: lustre-OST0000: new disk, initializing [ 6702.329648] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6702.332043] Lustre: Skipped 1 previous similar message [ 6702.356505] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6704.328194] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6705.150445] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 6705.154428] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 6705.164150] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 6920.160415] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6920.164784] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6921.611091] LustreError: 300514:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 6921.726465] Lustre: server umount lustre-MDT0000 complete [ 6923.062336] LustreError: 300515:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766419304 with bad export cookie 9671281899243851346 [ 6923.064683] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6923.066718] LustreError: 300515:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6925.100098] LustreError: 300716:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6925.101856] LustreError: 300716:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6925.141415] Lustre: server umount lustre-OST0000 complete [ 6933.492436] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_hostid [ 6937.094714] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 6940.963596] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6943.408320] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6945.453430] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6947.634881] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6949.962764] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6949.986105] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6950.066450] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6950.077847] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6950.111215] Lustre: lustre-MDT0000: new disk, initializing [ 6950.136813] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6950.144419] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6951.393800] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6955.483638] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6955.507841] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6955.540529] Lustre: 303755:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6955.562952] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6955.565083] Lustre: Skipped 1 previous similar message [ 6955.602163] Lustre: lustre-MDT0001: new disk, initializing [ 6955.630824] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6955.642552] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6955.647056] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6956.886251] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6959.839515] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6960.995562] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6963.245703] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6963.270546] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6963.361829] Lustre: lustre-OST0000: new disk, initializing [ 6963.363701] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6963.393428] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6965.248584] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 6965.940911] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6965.943522] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6965.972256] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6968.298698] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6969.648620] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 6969.737473] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 6971.064684] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 6971.141478] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 6976.482943] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6976.483482] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6976.486089] LustreError: Skipped 1 previous similar message [ 6976.486920] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6976.490069] Lustre: Skipped 1 previous similar message [ 6981.603349] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6981.605402] Lustre: Skipped 2 previous similar messages [ 6986.207102] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6986.224843] LustreError: 306694:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 6986.226539] LustreError: 306694:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6986.262513] Lustre: server umount lustre-OST0000 complete [ 6991.841618] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6991.841796] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6991.842741] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6991.850593] Lustre: Skipped 1 previous similar message [ 6993.660612] LustreError: 306897:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 6993.662539] LustreError: 306897:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6993.686911] Lustre: server umount lustre-MDT0000 complete [ 6994.941175] LustreError: 304536:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766419376 with bad export cookie 9671281899246792123 [ 6994.945039] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6994.945610] LustreError: 304536:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6995.051575] Lustre: server umount lustre-MDT0001 complete [ 6996.147625] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 6996.692203] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 11:02:59 (1766419379) [ 6998.885867] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6999.042269] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7000.318565] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7003.301309] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7004.834536] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7006.644438] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7007.808265] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7010.063589] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7012.079546] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7013.904288] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7016.132233] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7016.155879] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7016.204516] Lustre: lustre-OST0001: new disk, initializing [ 7016.206362] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7016.233835] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7016.236717] Lustre: Skipped 2 previous similar messages [ 7018.066069] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7025.149172] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7025.153707] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7025.169650] LustreError: 310736:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 7025.172684] LustreError: 310736:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 7026.209949] LustreError: 310736:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 7026.212413] LustreError: 310736:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 7026.929230] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7027.002151] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 7027.234460] LustreError: 310736:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 7027.238913] LustreError: 310736:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 7028.148249] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9a2d581fd000.ost_server_uuid 50 [ 7028.647295] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9a2d581fd000.ost_server_uuid in FULL state after 0 sec [ 7029.282436] LustreError: 310736:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 7029.286221] LustreError: 310736:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 7029.833689] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9a2d581fd000.ost_server_uuid 50 [ 7030.354936] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9a2d581fd000.ost_server_uuid in FULL state after 0 sec [ 7031.338911] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7045.602587] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7045.603673] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7045.606461] Lustre: Skipped 1 previous similar message [ 7045.608294] Lustre: Skipped 1 previous similar message [ 7055.842762] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7055.845103] Lustre: Skipped 5 previous similar messages [ 7056.351104] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7056.372138] LustreError: 311474:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 7056.374815] LustreError: 311474:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 7056.392551] Lustre: server umount lustre-OST0001 complete [ 7059.423748] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7059.426446] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7060.963144] LustreError: 309269:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7060.966910] LustreError: 309269:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7064.075282] Lustre: server umount lustre-OST0000 complete [ 7071.421565] LustreError: 311878:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 7071.424054] LustreError: 311878:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 7071.452498] Lustre: server umount lustre-MDT0000 complete [ 7072.737132] LustreError: 307695:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766419454 with bad export cookie 9671281899246792991 [ 7072.739784] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7072.742132] LustreError: 307695:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7072.869485] Lustre: server umount lustre-MDT0001 complete [ 7076.571369] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 7077.692810] Key type lgssc unregistered [ 7077.826368] LNet: 312611:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7077.829099] LNetError: 312611:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7077.836336] LNet: Removed LNI 192.168.204.145@tcp [ 7078.142161] Key type .llcrypt unregistered [ 7078.143708] Key type ._llcrypt unregistered [ 7087.914911] Key type ._llcrypt registered [ 7087.916370] Key type .llcrypt registered [ 7087.959838] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 7088.310402] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7088.350946] alg: No test for adler32 (adler32-zlib) [ 7089.213918] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 7089.304209] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 7090.887215] Key type lgssc registered [ 7091.317793] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7094.165449] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7094.173212] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7095.328563] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7096.644388] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7098.530815] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7100.724402] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7100.849605] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7102.170552] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7105.212089] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7105.331108] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7107.167922] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7108.973403] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7111.286733] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7111.357294] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7113.216904] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7115.026986] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 7119.593206] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7119.593328] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 7121.673909] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 7121.738385] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 7199.712414] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7199.716362] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7199.722729] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7201.764278] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7201.764884] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7201.765188] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7203.756317] LustreError: 317412:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 7203.771280] Lustre: server umount lustre-OST0001 complete [ 7206.881355] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7206.882589] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7206.885092] LustreError: 315541:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7206.885103] LustreError: 315541:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7206.885622] Lustre: Skipped 1 previous similar message [ 7206.885861] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7206.903288] Lustre: Skipped 2 previous similar messages [ 7211.371191] LustreError: 317613:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 7211.374173] LustreError: 317613:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7211.411070] Lustre: server umount lustre-OST0000 complete [ 7213.536120] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7213.538409] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7213.542650] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7218.749909] LustreError: 317814:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 7218.751904] LustreError: 317814:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7218.776981] Lustre: server umount lustre-MDT0000 complete [ 7220.133474] LustreError: 316449:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766419601 with bad export cookie 12381890574811649524 [ 7220.136615] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7220.138101] LustreError: 316449:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7220.283358] Lustre: server umount lustre-MDT0001 complete [ 7223.889905] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 7225.039860] Key type lgssc unregistered [ 7225.181590] LNet: 318547:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7225.184235] LNetError: 318547:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7225.191460] LNet: Removed LNI 192.168.204.145@tcp [ 7225.530518] Key type .llcrypt unregistered [ 7225.532069] Key type ._llcrypt unregistered [ 7243.556767] Key type ._llcrypt registered [ 7243.557828] Key type .llcrypt registered [ 7243.596864] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 7243.899973] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7243.941090] alg: No test for adler32 (adler32-zlib) [ 7244.788523] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 7244.876416] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 7246.455157] Key type lgssc registered [ 7246.843600] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7250.046533] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7250.055535] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7251.240745] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7252.692520] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7255.749523] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7255.905858] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7257.385576] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7258.500668] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7260.804688] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7260.929579] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7261.986936] LustreError: 322435:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7261.990418] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 7261.993899] LustreError: 322435:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7263.037838] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7266.088460] LustreError: 322252:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7266.096218] LustreError: 322252:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7266.317512] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7266.400691] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7268.485684] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7269.042896] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 7272.948811] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7277.722238] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7286.752174] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7286.755260] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7286.760570] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7287.265047] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7287.266773] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7287.271591] Lustre: Skipped 1 previous similar message [ 7292.384132] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7292.386495] Lustre: Skipped 6 previous similar messages [ 7292.414680] LustreError: 324206:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 7292.440161] Lustre: server umount lustre-MDT0000 complete [ 7293.573138] LustreError: 320777:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766419675 with bad export cookie 4037700765987156264 [ 7293.575570] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7293.575858] LustreError: 320777:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7293.641677] LustreError: 324408:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 7293.643408] LustreError: 324408:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 7293.689454] Lustre: server umount lustre-MDT0001 complete [ 7304.693275] LustreError: 324608:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 7304.695000] LustreError: 324608:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 7304.710601] Lustre: server umount lustre-OST0000 complete [ 7316.082438] LustreError: 324810:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 7316.085421] LustreError: 324810:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7316.131461] Lustre: server umount lustre-OST0001 complete [ 7321.086527] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 7324.310810] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7324.443682] LustreError: 325986:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7324.466304] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7325.808292] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7328.744236] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7330.236289] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7331.193698] Lustre: 327094:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7333.403321] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7333.516150] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7333.518045] Lustre: Skipped 1 previous similar message [ 7334.562642] LustreError: 327449:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7334.566434] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 7335.336009] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7338.060432] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7339.181675] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 7340.103096] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7343.081826] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7344.230832] Lustre: 328939:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7354.335952] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7354.339045] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7354.342505] Lustre: Skipped 1 previous similar message [ 7354.344076] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7356.221193] LustreError: 329043:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 7356.222926] LustreError: 329043:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7356.249919] Lustre: server umount lustre-MDT0000 complete [ 7357.367134] LustreError: 325965:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766419738 with bad export cookie 4037700765987157951 [ 7357.369607] LustreError: 325965:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7357.369991] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7357.481295] Lustre: server umount lustre-MDT0001 complete [ 7369.206196] LustreError: 329445:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 7369.210018] LustreError: 329445:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 7369.229089] Lustre: server umount lustre-OST0000 complete [ 7380.823651] Lustre: server umount lustre-OST0001 complete [ 7381.991455] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 7382.525525] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 7383.134717] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 11:09:25 (1766419765) [ 7391.053803] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 7394.328151] loop0: detected capacity change from 0 to 1099511627776 [ 7394.488744] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 7394.492276] *** VALIDATE xfs *** [ 7394.500677] XFS (loop0): Mounting V5 Filesystem [ 7394.512041] XFS (loop0): Ending clean mount [ 7394.967282] loop1: detected capacity change from 0 to 18691697672192 [ 7409.395995] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7414.800188] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_hostid [ 7417.960187] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 7421.041340] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7423.184583] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7425.225156] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7427.259939] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7429.421577] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7429.444182] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7429.520255] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7429.531953] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7429.567040] Lustre: lustre-MDT0000: new disk, initializing [ 7429.588100] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7429.589826] Lustre: Skipped 1 previous similar message [ 7429.594814] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7430.771568] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7434.866833] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7434.887775] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7434.924317] Lustre: 334011:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7434.946197] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7434.947984] Lustre: Skipped 1 previous similar message [ 7434.986175] Lustre: lustre-MDT0001: new disk, initializing [ 7435.037036] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7435.041589] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7436.464663] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7439.730864] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7441.003507] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7443.406977] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7443.432330] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7443.529699] Lustre: lustre-OST0000: new disk, initializing [ 7443.531543] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7444.732877] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7444.738075] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7444.780683] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7445.634689] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7448.862262] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7450.234962] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7450.309709] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 7451.579268] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 7451.649338] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 7454.317702] LustreError: 336953:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 7454.319741] LustreError: 336953:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 7454.363186] Lustre: server umount lustre-OST0000 complete [ 7455.711931] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7455.714951] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7455.718578] Lustre: Skipped 3 previous similar messages [ 7455.720619] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7455.722608] Lustre: Skipped 3 previous similar messages [ 7460.320693] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7465.439941] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7465.441674] Lustre: Skipped 3 previous similar messages [ 7466.490715] Lustre: server umount lustre-MDT0000 complete [ 7467.684965] LustreError: 334792:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766419849 with bad export cookie 4037700765987159505 [ 7467.688054] LustreError: 334792:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7467.688886] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7478.296793] Lustre: server umount lustre-MDT0001 complete [ 7481.129531] XFS (loop0): Unmounting Filesystem [ 7482.705302] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 11:11:05 (1766419865) [ 7484.958772] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7485.108884] LustreError: 338174:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7485.130079] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7485.132915] Lustre: Skipped 2 previous similar messages [ 7486.519271] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7489.464041] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7490.869840] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7492.656494] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7493.775139] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7496.057676] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7498.067536] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7499.861517] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7520.736747] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7520.739783] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7520.745591] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7524.782114] LustreError: 340536:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7524.784191] LustreError: 340536:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 7524.820340] Lustre: server umount lustre-OST0000 complete [ 7532.114420] Lustre: server umount lustre-MDT0000 complete [ 7533.302372] LustreError: 338892:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766419914 with bad export cookie 4037700765987160387 [ 7533.305192] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7533.306766] LustreError: 338892:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7533.436885] Lustre: server umount lustre-MDT0001 complete [ 7537.091170] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 7538.080366] Key type lgssc unregistered [ 7538.211531] LNet: 341471:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7538.214738] LNetError: 341471:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7538.220454] LNet: Removed LNI 192.168.204.145@tcp [ 7538.564133] Key type .llcrypt unregistered [ 7538.565127] Key type ._llcrypt unregistered [ 7548.352522] Key type ._llcrypt registered [ 7548.353464] Key type .llcrypt registered [ 7548.390440] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 7548.783744] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7548.789873] alg: No test for adler32 (adler32-zlib) [ 7549.677486] Lustre: Lustre: Build Version: 2.17.0_RC3_1_g3717bff [ 7549.773630] LNet: Added LNI 192.168.204.145@tcp [8/256/0/180] [ 7551.359166] Key type lgssc registered [ 7551.779087] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7554.640600] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7554.647669] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7555.809971] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7557.120711] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7559.921317] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7560.048783] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7561.345434] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7563.109486] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7564.325269] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7566.574696] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7566.684929] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7568.478636] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7570.149851] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7590.879840] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 7590.882410] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7590.886832] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7591.392372] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7591.394390] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7591.394836] Lustre: Skipped 1 previous similar message [ 7594.298868] LustreError: 345317:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7594.319746] Lustre: server umount lustre-MDT0001 complete [ 7595.239454] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7596.011416] LustreError: 342993:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.45@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7596.016810] LustreError: 342993:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7596.513720] LustreError: 342998:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7596.518764] LustreError: 342998:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7597.633688] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7597.660164] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 7597.661961] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7597.673513] Lustre: 342986:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7597.773506] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7598.970256] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7603.172718] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 7605.220474] LustreError: 346443:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 7605.223256] LustreError: 346443:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 7605.225526] LustreError: 342990:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 7605.240505] LustreError: 346446:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.204.145@tcp: cfg command failed: rc = -17 [ 7605.243365] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 7605.243365] [ 7903.218980] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.45@tcp (stopping) [ 7905.252218] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7905.252691] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 7905.252965] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7905.255742] Lustre: Skipped 1 previous similar message [ 7907.008541] LustreError: 357984:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7907.010671] LustreError: 357984:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7907.039243] Lustre: server umount lustre-MDT0001 complete [ 7908.101659] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7908.330962] LustreError: 353125:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.45@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7910.368900] LustreError: 346361:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7910.377089] LustreError: 346361:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7910.839439] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7910.873292] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 7910.874979] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7910.887878] Lustre: 342987:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7911.013377] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7912.474680] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 7916.518447] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 7919.079358] LustreError: 359128:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 7919.082637] LustreError: 359128:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 7919.086159] LustreError: 342990:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 7919.088866] LustreError: 342990:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 7919.108640] LustreError: 359141:0:(obd_config.c:689:class_setup()) Device 8 already setup (type osp) [ 7919.112552] LustreError: 359141:0:(obd_config.c:797:class_detach()) OBD device 8 still set up [ 7919.116052] LustreError: 359141:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.204.145@tcp: cfg command failed: rc = -17 [ 7919.120985] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.145@tcp [ 7919.120985] [ 8215.535550] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.45@tcp (stopping) [ 8215.537759] Lustre: Skipped 1 previous similar message [ 8218.593573] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 8218.594305] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8218.603374] Lustre: Skipped 1 previous similar message [ 8223.712387] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8223.714283] Lustre: Skipped 4 previous similar messages [ 8229.343147] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8229.386780] LustreError: 370748:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 8229.389838] LustreError: 370748:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8229.415060] Lustre: server umount lustre-MDT0001 complete [ 8230.378511] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8230.890427] LustreError: 342992:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.45@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8232.962964] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8233.001114] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 8233.002812] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8233.015187] Lustre: 342986:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8233.149834] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 8234.590622] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 8238.567989] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 8240.101725] LustreError: 371876:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 8240.104214] LustreError: 371876:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 8240.107867] LustreError: 342990:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 8240.112227] LustreError: 342990:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 8240.134419] LustreError: 371879:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 8240.142271] LustreError: 371879:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.204.145@tcp: cfg command failed: rc = -114 [ 8240.146351] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 8240.146351] [ 8550.881669] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8550.882449] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8550.884592] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8550.887543] Lustre: Skipped 1 previous similar message [ 8550.892992] Lustre: Skipped 5 previous similar messages [ 8554.751413] LustreError: 383386:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 8554.754340] LustreError: 383386:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8554.787213] Lustre: server umount lustre-MDT0000 complete [ 8556.000506] LustreError: 343708:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8556.006838] LustreError: 343708:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8556.027293] LustreError: 342978:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766420937 with bad export cookie 1494682486785552854 [ 8556.031052] LustreError: 342978:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8556.031440] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8561.120411] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8561.123786] Lustre: Skipped 1 previous similar message [ 8570.335100] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8570.373298] LustreError: 383587:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 8570.375101] LustreError: 383587:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8570.423631] Lustre: server umount lustre-MDT0001 complete [ 8571.802915] Lustre: server umount lustre-OST0000 complete [ 8574.953962] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_hostid [ 8578.230053] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 8581.783758] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8583.993222] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8586.028862] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8588.096604] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8590.418669] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 11:29:32 (1766420972) [ 8592.744248] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8592.771832] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8592.857508] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8592.868886] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8592.910094] Lustre: lustre-MDT0000: new disk, initializing [ 8592.938554] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8592.945251] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8594.275659] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 8598.568675] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8598.592847] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8598.629132] Lustre: 386320:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8598.657754] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8598.659496] Lustre: Skipped 1 previous similar message [ 8598.707885] Lustre: lustre-MDT0001: new disk, initializing [ 8598.746778] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8598.760066] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8598.763309] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8600.145794] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 8603.304747] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8604.437022] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8606.822168] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8606.850382] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8606.970588] Lustre: lustre-OST0000: new disk, initializing [ 8606.972923] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8607.014987] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8608.993156] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_default_debug -1 all [ 8612.143885] Lustre: DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8613.368989] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8613.372033] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8613.406507] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8623.585714] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8623.586776] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8623.592210] Lustre: Skipped 2 previous similar messages [ 8623.594788] Lustre: Skipped 3 previous similar messages [ 8625.986369] LustreError: 388765:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 8625.989048] LustreError: 388765:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 8626.024941] Lustre: server umount lustre-MDT0000 complete [ 8627.240203] LustreError: 387100:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1766421008 with bad export cookie 1494682486785554814 [ 8627.242455] LustreError: 387113:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 8627.243467] LustreError: MGC192.168.204.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8627.245474] LustreError: 387100:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8627.359759] Lustre: server umount lustre-MDT0001 complete [ 8638.877620] Lustre: server umount lustre-OST0000 complete [ 8648.113196] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing set_hostid [ 8651.178913] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing load_modules_local [ 8654.492094] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8656.691201] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8658.833203] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8660.951386] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8661.512108] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 8406 sec ========== 11:30:43 (1766421043) [ 8662.092919] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 11:30:44 (1766421044) === [ 8663.200485] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 11:30:45 (1766421045) === [ 8676.047556] Lustre: DEBUG MARKER: oleg445-server.virtnet: executing unload_modules_local [ 8677.109528] Key type lgssc unregistered [ 8677.239467] LNet: 394267:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8677.243237] LNetError: 394267:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8677.251329] LNet: Removed LNI 192.168.204.145@tcp [ 8677.577922] Key type .llcrypt unregistered [ 8677.579057] Key type ._llcrypt unregistered