[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 528726672 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f5b30-0x000f5b3f] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F5950 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE1BB7 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE1A53 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 001A13 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE1AC7 000090 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE1B57 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE1B8F 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe1a53-0xbffe1ac6] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe1a52] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe1ac7-0xbffe1b56] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe1b57-0xbffe1b8e] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe1b8f-0xbffe1bb6] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001015] APIC: Switch to symmetric I/O mode setup [ 0.002410] x2apic enabled [ 0.003012] Switched APIC routing to physical x2apic. [ 0.004025] kvm-guest: setup PV IPIs [ 0.007522] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008033] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009019] pid_max: default: 32768 minimum: 301 [ 0.010186] LSM: Security Framework initializing [ 0.012105] Yama: becoming mindful. [ 0.013075] SELinux: Initializing. [ 0.014000] *** VALIDATE selinux *** [ 0.023779] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028366] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029206] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031122] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032146] *** VALIDATE tmpfs *** [ 0.034096] *** VALIDATE proc *** [ 0.035276] *** VALIDATE cgroup *** [ 0.036015] *** VALIDATE cgroup2 *** [ 0.037322] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.038183] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.039015] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.040000] Spectre V2 : User space: Vulnerable [ 0.041016] Speculative Store Bypass: Vulnerable [ 0.044647] debug: unmapping init [mem 0xffffffff8dc59000-0xffffffff8dc60fff] [ 0.046890] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.048000] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.049037] ... version: 2 [ 0.050015] ... bit width: 48 [ 0.051013] ... generic registers: 4 [ 0.052015] ... value mask: 0000ffffffffffff [ 0.053018] ... max period: 00007fffffffffff [ 0.054017] ... fixed-purpose events: 3 [ 0.055016] ... event mask: 000000070000000f [ 0.056403] rcu: Hierarchical SRCU implementation. [ 0.058795] smp: Bringing up secondary CPUs ... [ 0.059667] x86: Booting SMP configuration: [ 0.060033] .... node #0, CPUs: #1 #2 #3 [ 0.073121] smp: Brought up 1 node, 4 CPUs [ 0.075017] smpboot: Max logical packages: 1 [ 0.076022] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.238024] node 0 deferred pages initialised in 159ms [ 0.241137] devtmpfs: initialized [ 0.242269] x86/mm: Memory block size: 128MB [ 0.245573] gcov: version magic: 0x41383552 [ 0.247366] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.248099] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.249269] pinctrl core: initialized pinctrl subsystem [ 0.250209] [ 0.250756] ************************************************************* [ 0.251019] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.252012] ** ** [ 0.253015] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.254012] ** ** [ 0.255021] ** This means that this kernel is built to expose internal ** [ 0.256016] ** IOMMU data structures, which may compromise security on ** [ 0.257015] ** your system. ** [ 0.258017] ** ** [ 0.259014] ** If you see this message and you are not debugging the ** [ 0.260014] ** kernel, report this immediately to your vendor! ** [ 0.261013] ** ** [ 0.262015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.263021] ************************************************************* [ 0.264813] NET: Registered protocol family 16 [ 0.265449] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.266079] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.267068] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.268468] cpuidle: using governor menu [ 0.272527] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.274534] PCI: Using configuration type 1 for base access [ 0.277125] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.291280] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.293932] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.298643] cryptd: max_cpu_qlen set to 1000 [ 0.302621] ACPI: Added _OSI(Module Device) [ 0.303000] ACPI: Added _OSI(Processor Device) [ 0.303000] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.303000] ACPI: Added _OSI(Processor Aggregator Device) [ 0.303000] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.305622] ACPI: Interpreter enabled [ 0.306056] ACPI: PM: (supports S0 S3 S4 S5) [ 0.307013] ACPI: Using IOAPIC for interrupt routing [ 0.308157] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.309396] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.318427] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.319041] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.320023] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.321092] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.323267] acpiphp: Slot [2] registered [ 0.324161] acpiphp: Slot [3] registered [ 0.325079] acpiphp: Slot [4] registered [ 0.326083] acpiphp: Slot [5] registered [ 0.327167] acpiphp: Slot [6] registered [ 0.328233] acpiphp: Slot [7] registered [ 0.329115] acpiphp: Slot [8] registered [ 0.330114] acpiphp: Slot [9] registered [ 0.331546] acpiphp: Slot [10] registered [ 0.332127] acpiphp: Slot [11] registered [ 0.333082] acpiphp: Slot [12] registered [ 0.334123] acpiphp: Slot [13] registered [ 0.335081] acpiphp: Slot [14] registered [ 0.336098] acpiphp: Slot [15] registered [ 0.337112] acpiphp: Slot [16] registered [ 0.338093] acpiphp: Slot [17] registered [ 0.339143] acpiphp: Slot [18] registered [ 0.340102] acpiphp: Slot [19] registered [ 0.341080] acpiphp: Slot [20] registered [ 0.342186] acpiphp: Slot [21] registered [ 0.343134] acpiphp: Slot [22] registered [ 0.344097] acpiphp: Slot [23] registered [ 0.345098] acpiphp: Slot [24] registered [ 0.346118] acpiphp: Slot [25] registered [ 0.347092] acpiphp: Slot [26] registered [ 0.348092] acpiphp: Slot [27] registered [ 0.349101] acpiphp: Slot [28] registered [ 0.350115] acpiphp: Slot [29] registered [ 0.351094] acpiphp: Slot [30] registered [ 0.352089] acpiphp: Slot [31] registered [ 0.353056] PCI host bridge to bus 0000:00 [ 0.354014] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.355017] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.356021] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.357020] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.358027] pci_bus 0000:00: root bus resource [mem 0x180000000-0x1ffffffff window] [ 0.359024] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.360591] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.363610] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.366495] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.372608] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.375723] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.376028] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.377026] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.378024] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.379614] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.380799] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.381062] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.382755] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.384897] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.390022] pci 0000:00:02.0: reg 0x20: [mem 0xfebe4000-0xfebe7fff 64bit pref] [ 0.392815] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.396898] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.409020] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.421028] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.446027] pci 0000:00:05.0: reg 0x20: [mem 0xfebe8000-0xfebebfff 64bit pref] [ 0.457379] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.471017] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.483021] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.509017] pci 0000:00:06.0: reg 0x20: [mem 0xfebec000-0xfebeffff 64bit pref] [ 0.524034] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.532021] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.540023] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.556018] pci 0000:00:07.0: reg 0x20: [mem 0xfebf0000-0xfebf3fff 64bit pref] [ 0.567708] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.577017] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.584018] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.599016] pci 0000:00:08.0: reg 0x20: [mem 0xfebf4000-0xfebf7fff 64bit pref] [ 0.609509] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.620015] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.625014] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.638018] pci 0000:00:09.0: reg 0x20: [mem 0xfebf8000-0xfebfbfff 64bit pref] [ 0.648812] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.658018] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.668018] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.683018] pci 0000:00:0a.0: reg 0x20: [mem 0xfebfc000-0xfebfffff 64bit pref] [ 0.698352] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.701365] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.704400] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.706407] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.709221] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.715023] iommu: Default domain type: Passthrough [ 0.716402] SCSI subsystem initialized [ 0.717000] ACPI: bus type USB registered [ 0.717000] usbcore: registered new interface driver usbfs [ 0.718138] usbcore: registered new interface driver hub [ 0.721078] usbcore: registered new device driver usb [ 0.723163] pps_core: LinuxPPS API ver. 1 registered [ 0.725012] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.728651] PTP clock support registered [ 0.732009] EDAC MC: Ver: 3.0.0 [ 0.735158] PCI: Using ACPI for IRQ routing [ 0.736813] NetLabel: Initializing [ 0.737008] NetLabel: domain hash size = 128 [ 0.738007] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.739610] NetLabel: unlabeled traffic allowed by default [ 0.740000] vgaarb: loaded [ 0.742287] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.744010] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.753877] clocksource: Switched to clocksource kvm-clock [ 0.868908] VFS: Disk quotas dquot_6.6.0 [ 0.870614] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.872851] *** VALIDATE ramfs *** [ 0.874296] *** VALIDATE hugetlbfs *** [ 0.875641] pnp: PnP ACPI init [ 0.878913] pnp: PnP ACPI: found 6 devices [ 0.899410] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.902562] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.905891] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.909348] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.911173] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.913832] pci_bus 0000:00: resource 8 [mem 0x180000000-0x1ffffffff window] [ 0.915766] NET: Registered protocol family 2 [ 0.917948] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.922928] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.926347] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.932036] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.935725] TCP: Hash tables configured (established 65536 bind 65536) [ 0.937827] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.940348] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.942574] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.945690] NET: Registered protocol family 1 [ 0.947938] RPC: Registered named UNIX socket transport module. [ 0.950228] RPC: Registered udp transport module. [ 0.951595] RPC: Registered tcp transport module. [ 0.952821] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.954887] NET: Registered protocol family 44 [ 0.956668] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.959219] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.961597] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.963548] PCI: CLS 0 bytes, default 64 [ 0.965479] Unpacking initramfs... [ 2.591701] debug: unmapping init [mem 0xffff9dec7cc54000-0xffff9dec7ffbffff] [ 2.597683] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.599844] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.603796] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.127062] Initialise system trusted keyrings [ 3.128713] Key type blacklist registered [ 3.132255] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.141640] zbud: loaded [ 3.144979] *** VALIDATE nfs *** [ 3.146042] *** VALIDATE nfs4 *** [ 3.147347] pstore: using deflate compression [ 3.150746] Platform Keyring initialized [ 3.298704] NET: Registered protocol family 38 [ 3.300368] Key type asymmetric registered [ 3.301916] Asymmetric key parser 'x509' registered [ 3.305104] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.308495] io scheduler mq-deadline registered [ 3.310856] io scheduler kyber registered [ 3.312912] io scheduler bfq registered [ 3.314550] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.317548] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.320166] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.331330] ACPI: Power Button [PWRF] [ 3.441330] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.543026] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.748428] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.853501] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 4.092483] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 4.126844] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 4.167315] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 4.174976] Non-volatile memory driver v1.3 [ 4.179811] Linux agpgart interface v0.103 [ 4.225270] virtio_blk virtio1: [vda] 133856 512-byte logical blocks (68.5 MB/65.4 MiB) [ 4.229696] vda: detected capacity change from 0 to 68534272 [ 4.248220] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 4.253145] vdb: detected capacity change from 0 to 1073741824 [ 4.273248] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.277272] vdc: detected capacity change from 0 to 2621440000 [ 4.295140] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.299346] vdd: detected capacity change from 0 to 2621440000 [ 4.319497] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.322361] vde: detected capacity change from 0 to 4294967296 [ 4.346955] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.350085] vdf: detected capacity change from 0 to 4294967296 [ 4.360656] libphy: Fixed MDIO Bus: probed [ 4.369326] usbcore: registered new interface driver usbserial_generic [ 4.371620] usbserial: USB Serial support registered for generic [ 4.376555] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 4.385562] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 4.393545] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 4.396611] mousedev: PS/2 mouse device common for all mice [ 4.403411] rtc_cmos 00:05: RTC can wake from S4 [ 4.407388] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 4.418642] rtc_cmos 00:05: registered as rtc0 [ 4.420495] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 4.423539] intel_pstate: CPU model not supported [ 4.428677] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 4.434029] hid: raw HID events driver (C) Jiri Kosina [ 4.440271] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 4.444565] usbcore: registered new interface driver usbhid [ 4.446718] usbhid: USB HID core driver [ 4.448350] drop_monitor: Initializing network drop monitor service [ 4.452359] Initializing XFRM netlink socket [ 4.454392] NET: Registered protocol family 10 [ 4.459104] Segment Routing with IPv6 [ 4.460853] NET: Registered protocol family 17 [ 4.462941] mpls_gso: MPLS GSO support [ 4.469365] RAS: Correctable Errors collector initialized. [ 4.471590] AVX version of gcm_enc/dec engaged. [ 4.473354] AES CTR mode by8 optimization enabled [ 4.576519] sched_clock: Marking stable (4576472056, 0)->(5811593197, -1235121141) [ 4.581596] registered taskstats version 1 [ 4.585603] Loading compiled-in X.509 certificates [ 4.587450] zswap: loaded using pool lzo/zbud [ 4.633890] Key type big_key registered [ 4.653837] Key type encrypted registered [ 4.655328] ima: No TPM chip found, activating TPM-bypass! [ 4.658530] ima: Allocated hash algorithm: sha1 [ 4.660479] ima: No architecture policies found [ 4.662339] evm: Initialising EVM extended attributes: [ 4.664378] evm: security.selinux [ 4.665733] evm: security.ima [ 4.667054] evm: security.capability [ 4.668583] evm: HMAC attrs: 0x1 [ 4.671194] rtc_cmos 00:05: setting system clock to 2025-11-18 23:53:52 UTC (1763510032) [ 4.676930] debug: unmapping init [mem 0xffffffff8ec03000-0xffffffff8edfffff] [ 4.680149] debug: unmapping init [mem 0xffffffff8d982000-0xffffffff8dc58fff] [ 4.692113] Write protecting the kernel read-only data: 28672k [ 4.699707] debug: unmapping init [mem 0xffffffff8c003000-0xffffffff8c1fffff] [ 4.704841] debug: unmapping init [mem 0xffffffff8c914000-0xffffffff8c9fffff] [ 4.782490] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.790595] systemd[1]: Detected virtualization kvm. [ 4.792367] systemd[1]: Detected architecture x86-64. [ 4.793982] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.823193] systemd[1]: No hostname configured. [ 4.824983] systemd[1]: Set hostname to . [ 4.827869] random: systemd: uninitialized urandom read (16 bytes read) [ 4.830529] systemd[1]: Initializing machine ID from random generator. [ 4.883792] random: ln: uninitialized urandom read (6 bytes read) [ 4.976970] random: systemd: uninitialized urandom read (16 bytes read) [ 4.981331] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 4.992209] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 4.996327] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Initrd Root Device. [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Swap. Starting Setup Virtual Console... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.833157] device-mapper: uevent: version 1.0.3 [ 5.836928] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 6.688321] virtio_net virtio0 ens2: renamed from eth0 [ 6.748541] random: fast init done [ 6.934782] scsi host0: ata_piix [ 6.968259] scsi host1: ata_piix [ 6.970345] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 6.973217] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 11.174991] random: crng init done [ 11.177642] random: 7 urandom warning(s) missed due to ratelimiting [ 11.696178] dracut-initqueue[586]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 12.588232] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 13.927270] printk: systemd: 25 output lines suppressed due to ratelimiting [ 14.199510] SELinux: Disabled at runtime. [ 14.260317] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 14.268928] systemd[1]: Detected virtualization kvm. [ 14.271015] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 14.863591] systemd[1]: initrd-switch-root.service: Succeeded. [ 14.866819] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 14.871497] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 14.876605] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 14.879931] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 14.893232] systemd[1]: Starting Journal Service... Starting Journal Service... [ 14.900387] systemd[1]: Mounting Kernel Debug File System... Mounting Kernel Debug File System... Mounting POSIX Message Queue File System... [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-getty.slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on udev Control Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Mounting Huge Pages File System... Starting Apply Kernel Variables... [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Created slice system-serial\x2dgetty.slice. Starting Remount Root and Kernel File Systems... [ OK ] Reached target Paths. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target rpc_pipefs.target. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ 15.103388] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 15.435295] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 15.825072] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 15.864869] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 15.998801] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 16.011077] EDAC sbridge: Ver: 1.1.2 [ 17.840505] Key type dns_resolver registered [ 18.149670] NFS: Registering the id_resolver key type [ 18.151468] Key type id_resolver registered [ 18.153071] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. [ OK ] Reached target sshd-keygen.target. Starting Restore /run/initramfs on shutdown... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started irqbalance daemon. Starting Login Service... Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting Network Manager Wait Online... [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg444-server login: [ 48.555421] libcfs: loading out-of-tree module taints kernel. [ 48.588159] Key type ._llcrypt registered [ 48.589608] Key type .llcrypt registered [ 48.677215] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 61.416313] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 62.325175] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 62.339949] alg: No test for adler32 (adler32-zlib) [ 63.662872] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 64.333732] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 66.035160] Key type lgssc registered [ 67.322102] Lustre: Echo OBD driver; http://www.lustre.org/ [ 78.805292] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 79.456923] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 85.474838] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 91.971525] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 98.958214] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 103.804753] hrtimer: interrupt took 5743652 ns [ 110.801281] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 120.627319] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 120.696343] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 120.708674] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 121.907092] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 121.939847] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 121.991607] Lustre: lustre-MDT0000: new disk, initializing [ 122.065935] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 122.088661] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 124.823117] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 134.469613] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 134.533959] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 134.600258] Lustre: 6520:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 134.631439] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 134.636233] Lustre: Skipped 1 previous similar message [ 134.685893] Lustre: lustre-MDT0001: new disk, initializing [ 134.765350] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 134.796249] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 134.802887] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 137.661244] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 141.605509] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 149.075168] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 149.127715] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 149.265803] Lustre: lustre-OST0000: new disk, initializing [ 149.269778] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 149.320273] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 153.209157] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 154.635262] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 154.640524] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 154.700703] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 164.189984] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 164.238666] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 164.305388] Lustre: lustre-OST0001: new disk, initializing [ 164.311483] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 164.368095] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 168.205647] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 169.488703] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 169.497486] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 169.576793] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 176.954651] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 181.264622] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 190.087170] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing check_logdir /tmp/testlogs/ [ 194.348569] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing yml_node [ 198.896853] Lustre: DEBUG MARKER: Client: 2.16.61.43 [ 201.301243] Lustre: DEBUG MARKER: MDS: 2.16.61.43 [ 203.342537] Lustre: DEBUG MARKER: OSS: 2.16.61.43 [ 204.607526] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Nov 18 18:57:10 EST 2025 [ 221.101166] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 222.549136] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 237.535735] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 237.538664] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 237.574607] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 241.122455] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 241.130544] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 241.135180] Lustre: Skipped 1 previous similar message [ 241.149039] Lustre: Skipped 1 previous similar message [ 243.528523] LustreError: 12225:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 243.813817] Lustre: server umount lustre-MDT0000 complete [ 246.545815] LustreError: 10288:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763510274 with bad export cookie 13133611276310112188 [ 246.554362] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 246.559972] LustreError: 10288:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 251.359836] LustreError: 6527:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 251.361111] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 251.361955] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 251.361959] Lustre: Skipped 2 previous similar messages [ 251.372338] LustreError: 6527:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 251.390615] Lustre: Skipped 2 previous similar messages [ 252.688590] LustreError: 12426:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 252.691153] LustreError: 12426:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 252.793760] Lustre: server umount lustre-MDT0001 complete [ 255.283126] LustreError: 12628:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 255.287057] LustreError: 12628:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 255.319086] Lustre: server umount lustre-OST0000 complete [ 257.772561] LustreError: 12829:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 257.775138] LustreError: 12829:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 257.868808] Lustre: server umount lustre-OST0001 complete [ 261.853659] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 268.390385] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 275.681553] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 280.251076] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 284.930650] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 289.345295] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 293.763807] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 293.802822] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 293.979972] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 294.009087] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 294.055146] Lustre: lustre-MDT0000: new disk, initializing [ 294.145111] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 294.165374] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 296.803715] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 305.926659] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 306.005484] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 306.147506] Lustre: 15009:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 306.239725] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 306.249713] Lustre: Skipped 1 previous similar message [ 306.338355] Lustre: lustre-MDT0001: new disk, initializing [ 306.434991] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 306.476046] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 306.491472] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 309.769265] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 318.331822] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 322.317570] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 329.017796] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 329.061993] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 329.239485] Lustre: lustre-OST0000: new disk, initializing [ 329.246477] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 329.342334] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 330.461121] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 330.472412] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 330.590761] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 333.985384] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 340.488533] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 344.046200] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 344.236992] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 347.189172] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 347.367678] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 351.201137] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 351.206456] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 351.208977] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 351.212761] Lustre: Skipped 1 previous similar message [ 351.217083] Lustre: Skipped 1 previous similar message [ 354.871106] LustreError: 17951:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 354.881958] LustreError: 17951:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 354.994781] Lustre: server umount lustre-OST0000 complete [ 361.441569] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 361.445258] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 361.457154] Lustre: Skipped 1 previous similar message [ 364.567607] LustreError: 18154:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 364.570458] LustreError: 18154:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 364.665594] Lustre: server umount lustre-MDT0000 complete [ 366.560292] LustreError: 16150:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 366.577293] LustreError: 16150:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 367.401115] LustreError: 15791:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763510395 with bad export cookie 13133611276310113721 [ 367.405837] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 367.409454] LustreError: 15791:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 367.639818] Lustre: server umount lustre-MDT0001 complete [ 371.971801] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 18:59:57 (1763510397) [ 377.772815] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 378.077754] LustreError: 18882:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 378.134911] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 381.307182] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 383.462612] LustreError: 18882:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 387.554109] LustreError: 18882:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 388.680281] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 392.503915] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 396.932973] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 399.741804] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 405.537510] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 410.748256] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 416.778199] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 426.256994] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 432.611828] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 432.613626] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 432.655614] Lustre: Skipped 2 previous similar messages [ 432.660060] Lustre: Skipped 3 previous similar messages [ 437.881923] LustreError: 21185:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 437.905601] LustreError: 21185:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 438.032388] Lustre: server umount lustre-MDT0000 complete [ 441.311221] LustreError: 18861:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763510469 with bad export cookie 13133611276310114603 [ 441.323229] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 441.326683] LustreError: 18861:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 441.627166] Lustre: server umount lustre-MDT0001 complete [ 455.654862] Lustre: server umount lustre-OST0000 complete [ 472.563644] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 480.095995] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 480.424792] LustreError: 22926:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 480.496909] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 480.502091] Lustre: Skipped 2 previous similar messages [ 483.488967] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 490.067691] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 493.328778] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 495.541627] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 501.168981] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 506.212524] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 513.334595] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 513.395776] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 513.520557] Lustre: lustre-OST0001: new disk, initializing [ 513.523967] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 515.463970] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 515.483117] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 515.578414] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 518.735256] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 527.314573] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 530.449211] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 546.787701] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 546.795165] LustreError: Skipped 1 previous similar message [ 546.799113] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 546.812061] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 546.818436] Lustre: Skipped 3 previous similar messages [ 550.075368] LustreError: 26407:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 550.106320] LustreError: 26407:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 550.197403] Lustre: server umount lustre-MDT0000 complete [ 552.419043] LustreError: 22926:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 552.428846] LustreError: 22926:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 553.027124] LustreError: 22909:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763510580 with bad export cookie 13133611276310115450 [ 553.033109] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 553.431539] Lustre: server umount lustre-MDT0001 complete [ 566.832185] Lustre: server umount lustre-OST0000 complete [ 580.260431] Lustre: server umount lustre-OST0001 complete [ 594.692681] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 603.878401] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 604.242360] LustreError: 28185:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 607.317257] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 613.833742] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 614.187344] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 614.190564] Lustre: Skipped 4 previous similar messages [ 617.295799] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 619.676809] Lustre: 29299:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 625.371886] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 630.714840] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 637.754322] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 642.340451] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 648.794957] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 652.236968] Lustre: 31154:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 664.547323] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 664.553709] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 664.559286] Lustre: Skipped 5 previous similar messages [ 664.575038] Lustre: Skipped 7 previous similar messages [ 668.105803] LustreError: 31359:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 668.108548] LustreError: 31359:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 668.216308] Lustre: server umount lustre-MDT0000 complete [ 669.679284] LustreError: 28181:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 669.688821] LustreError: 28181:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 671.320766] LustreError: 28165:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763510699 with bad export cookie 13133611276310116745 [ 671.323048] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 671.332603] LustreError: 28165:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 671.570710] Lustre: server umount lustre-MDT0001 complete [ 685.139659] Lustre: server umount lustre-OST0000 complete [ 697.526893] Lustre: server umount lustre-OST0001 complete [ 704.851616] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 19:05:30 (1763510730) [ 727.139833] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 735.677435] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 739.926441] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 747.173718] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 750.710737] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 752.885449] Lustre: 35060:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 758.050156] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 762.331445] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 769.600260] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 774.110969] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 780.218765] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 783.742885] Lustre: 36915:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 793.572098] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 793.590358] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 793.594843] Lustre: Skipped 3 previous similar messages [ 793.597823] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 793.619918] Lustre: Skipped 3 previous similar messages [ 799.390313] LustreError: 37067:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 799.393471] LustreError: 37067:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 799.473280] Lustre: server umount lustre-MDT0000 complete [ 802.698410] LustreError: 35063:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763510830 with bad export cookie 13133611276310117851 [ 802.701075] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 802.723887] LustreError: 35063:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 803.067775] Lustre: server umount lustre-MDT0001 complete [ 817.060429] Lustre: server umount lustre-OST0000 complete [ 830.358503] Lustre: server umount lustre-OST0001 complete [ 838.490633] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 19:07:44 (1763510864) [ 854.775409] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 863.939108] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 864.244281] LustreError: 39041:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 864.249964] LustreError: 39041:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 867.255194] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 874.865531] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 875.194083] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 875.202482] Lustre: Skipped 7 previous similar messages [ 877.963098] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 880.192498] Lustre: 40157:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 885.918924] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 890.522702] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 896.349555] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 900.444947] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 906.694438] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 915.499772] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 915.505337] Lustre: Skipped 1 previous similar message [ 922.925587] Lustre: lustre-MDT0000: Not available for connect from 192.168.204.44@tcp (stopping) [ 922.935589] Lustre: Skipped 8 previous similar messages [ 924.297362] Lustre: server umount lustre-MDT0000 complete [ 927.727503] LustreError: 39020:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763510955 with bad export cookie 13133611276310118957 [ 927.731653] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 927.740607] LustreError: 39020:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 930.787876] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 930.799482] Lustre: Skipped 5 previous similar messages [ 942.047139] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 942.215326] Lustre: server umount lustre-MDT0001 complete [ 948.872074] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 955.567842] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 961.985273] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 965.548660] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 967.531294] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 970.089678] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 971.595701] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 974.907693] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 976.177672] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 983.017325] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 985.859296] Lustre: server umount lustre-MDT0000 complete [ 988.526380] LustreError: 39022:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763511016 with bad export cookie 13133611276310120112 [ 988.530594] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 988.536901] LustreError: 39022:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 988.818479] Lustre: server umount lustre-MDT0001 complete [ 1001.526378] Lustre: server umount lustre-OST0000 complete [ 1013.449687] Lustre: server umount lustre-OST0001 complete [ 1020.692660] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 19:10:46 (1763511046) [ 1036.585189] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 1046.205191] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1049.835111] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1057.149303] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1060.173032] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1062.305385] Lustre: 47515:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1062.317889] Lustre: 47515:0:(mgs_llog.c:1346:mgs_modify_param()) Skipped 1 previous similar message [ 1067.062901] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1071.496664] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1079.407851] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1086.117898] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1093.928618] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1102.677782] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1113.569127] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1117.388404] LustreError: 49594:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 1117.392439] LustreError: 49594:0:(obd_class.h:479:obd_check_dev()) Skipped 67 previous similar messages [ 1117.445099] Lustre: server umount lustre-MDT0000 complete [ 1120.290224] LustreError: 46376:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763511148 with bad export cookie 13133611276310120924 [ 1120.297987] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1120.567565] Lustre: server umount lustre-MDT0001 complete [ 1133.964596] Lustre: server umount lustre-OST0000 complete [ 1137.120106] Lustre: 3667:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1763511148/real 1763511148] req@ffff9debc45fdc00 x1849174358689152/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1763511164 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1137.396348] Lustre: server umount lustre-OST0001 complete [ 1146.100089] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 19:12:52 (1763511172) [ 1148.144116] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1149.787974] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 19:12:55 (1763511175) [ 1159.244152] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 1166.535520] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 1174.659181] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1179.534947] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1184.811509] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1184.857568] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1185.051589] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1185.151646] Lustre: lustre-MDT0000: new disk, initializing [ 1185.251490] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1188.715302] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1195.815331] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1202.389929] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1202.451412] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1204.753262] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1204.784237] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1207.937189] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1215.203226] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1217.373823] Lustre: DEBUG MARKER: create test files [ 1245.667294] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1245.678636] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1245.697019] Lustre: Skipped 10 previous similar messages [ 1245.706794] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1245.709461] Lustre: Skipped 18 previous similar messages [ 1250.176996] Lustre: server umount lustre-OST0000 complete [ 1253.681398] Lustre: server umount lustre-MDT0000 complete [ 1263.664653] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 1266.300893] Key type lgssc unregistered [ 1266.559078] LNet: 54929:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1266.568267] LNetError: 54929:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1267.629251] LNet: Removed LNI 192.168.204.144@tcp [ 1268.670528] Key type .llcrypt unregistered [ 1268.671842] Key type ._llcrypt unregistered [ 1294.361596] Key type ._llcrypt registered [ 1294.363667] Key type .llcrypt registered [ 1294.523677] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 1295.901216] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1295.972904] alg: No test for adler32 (adler32-zlib) [ 1297.176716] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 1297.458318] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 1299.191172] Key type lgssc registered [ 1300.696201] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1307.982615] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1308.016749] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1309.547704] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1312.766454] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1317.333477] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1323.636476] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1323.928992] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1329.104593] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1333.625825] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1334.190597] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1334.236833] Lustre: 57531:0:(osd_internal.h:1472:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1334.243088] Lustre: 57531:0:(osd_handler.c:2082:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1334.247696] Lustre: 57531:0:(osd_handler.c:2089:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1334.254607] Lustre: 57531:0:(osd_handler.c:2099:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1334.258126] Lustre: 57531:0:(osd_handler.c:2106:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1334.268583] Lustre: 57531:0:(osd_handler.c:2113:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1343.254830] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1348.379543] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1354.721902] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1354.725471] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1354.750081] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1355.837201] LustreError: 58279:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1355.973329] Lustre: server umount lustre-OST0000 complete [ 1358.784051] LustreError: 58479:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1358.788864] LustreError: 58479:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1358.884872] Lustre: server umount lustre-MDT0000 complete [ 1367.104766] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 1369.619731] Key type lgssc unregistered [ 1369.843961] LNet: 59010:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1369.852118] LNetError: 59010:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1370.930865] LNet: Removed LNI 192.168.204.144@tcp [ 1371.690647] Key type .llcrypt unregistered [ 1371.693696] Key type ._llcrypt unregistered [ 1394.922792] Key type ._llcrypt registered [ 1394.924463] Key type .llcrypt registered [ 1395.010942] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 1395.649537] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1395.703734] alg: No test for adler32 (adler32-zlib) [ 1396.766867] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 1396.966658] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 1398.639275] Key type lgssc registered [ 1399.686903] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1406.449503] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1406.475757] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1407.808674] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1410.924233] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1417.242419] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1425.599776] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1426.104947] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1432.235237] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1432.239705] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1432.371317] Lustre: 61695:0:(osd_internal.h:1472:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1432.375618] Lustre: 61695:0:(osd_handler.c:2082:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1432.379483] Lustre: 61695:0:(osd_handler.c:2089:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1432.411886] Lustre: 61695:0:(osd_handler.c:2099:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1432.416310] Lustre: 61695:0:(osd_handler.c:2106:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1432.429383] Lustre: 61695:0:(osd_handler.c:2113:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1437.849391] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1443.457824] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1453.025863] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1453.038250] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1453.074752] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1458.148075] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1458.299267] LustreError: 62387:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1458.419277] Lustre: server umount lustre-OST0000 complete [ 1463.184347] LustreError: 62589:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1463.193444] LustreError: 62589:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1463.425206] Lustre: server umount lustre-MDT0000 complete [ 1472.138400] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 1473.883365] Key type lgssc unregistered [ 1474.100585] LNet: 63121:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1474.108731] LNetError: 63121:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1475.173434] LNet: Removed LNI 192.168.204.144@tcp [ 1475.962777] Key type .llcrypt unregistered [ 1475.964831] Key type ._llcrypt unregistered [ 1489.857321] Key type ._llcrypt registered [ 1489.859284] Key type .llcrypt registered [ 1489.934413] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 1502.553825] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 1503.337621] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1503.390645] alg: No test for adler32 (adler32-zlib) [ 1504.464752] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 1504.709652] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 1506.367751] Key type lgssc registered [ 1507.184575] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1514.917199] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1520.328087] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1526.976985] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1533.705643] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1540.090647] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1540.143221] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1540.156293] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1541.326517] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1541.363120] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1541.421217] Lustre: lustre-MDT0000: new disk, initializing [ 1541.537059] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1541.560341] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1544.846778] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1553.987808] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1554.103666] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1554.214063] Lustre: 66241:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1554.279977] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1554.284186] Lustre: Skipped 1 previous similar message [ 1554.378406] Lustre: lustre-MDT0001: new disk, initializing [ 1554.460325] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1554.489117] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1554.499268] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1557.661292] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1564.329229] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1567.164212] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1572.732635] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1572.790260] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1572.972841] Lustre: lustre-OST0000: new disk, initializing [ 1572.975851] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1573.062654] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1577.112169] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1580.074243] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1580.080775] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1580.193797] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1582.921448] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1585.771664] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1585.949956] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1588.986901] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1589.133826] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1590.755991] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1590.758748] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1590.759563] LustreError: Skipped 1 previous similar message [ 1590.761964] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1590.771918] Lustre: Skipped 1 previous similar message [ 1590.780503] Lustre: Skipped 1 previous similar message [ 1595.876056] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1595.885877] Lustre: Skipped 1 previous similar message [ 1596.558125] LustreError: 69177:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1596.653407] Lustre: server umount lustre-OST0000 complete [ 1600.480815] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1600.485377] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1600.497057] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1605.786691] LustreError: 69379:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1605.789236] LustreError: 69379:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1605.900438] Lustre: server umount lustre-MDT0000 complete [ 1609.482273] LustreError: 66231:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763511637 with bad export cookie 15977939306052029816 [ 1609.485070] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1609.494135] LustreError: 66231:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1609.636266] LustreError: 69581:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1609.644100] LustreError: 69581:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1609.794277] Lustre: server umount lustre-MDT0001 complete [ 1618.107872] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 19:20:43 (1763511643) [ 1628.569522] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 1641.665644] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1665.106310] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 1673.152338] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 1681.994197] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1688.532510] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1694.918275] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1702.283193] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1707.590747] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1707.644572] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1707.817582] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1707.859529] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1707.921440] Lustre: lustre-MDT0000: new disk, initializing [ 1708.034633] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1708.051550] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1711.007467] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1720.831990] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1720.917303] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1721.055598] Lustre: 73822:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1721.129726] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1721.132522] Lustre: Skipped 1 previous similar message [ 1721.188520] Lustre: lustre-MDT0001: new disk, initializing [ 1721.307227] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1721.358383] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1721.377411] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1725.348883] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1733.331035] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1737.677188] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1744.676490] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1744.744770] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1745.048364] Lustre: lustre-OST0000: new disk, initializing [ 1745.065591] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1745.141610] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1750.802513] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1752.606030] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1752.630272] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1752.727738] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1757.997522] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1763.469356] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1763.737740] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1768.066918] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1768.234414] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1773.543138] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1773.547870] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1773.548343] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1773.567016] Lustre: Skipped 2 previous similar messages [ 1773.574593] Lustre: Skipped 1 previous similar message [ 1778.657105] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1778.659720] Lustre: Skipped 1 previous similar message [ 1784.287661] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1784.322061] LustreError: 76758:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1784.326177] LustreError: 76758:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1784.416360] Lustre: server umount lustre-OST0000 complete [ 1788.896394] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1788.902385] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1788.912535] Lustre: Skipped 1 previous similar message [ 1788.925053] Lustre: Skipped 4 previous similar messages [ 1802.207278] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1802.287472] LustreError: 76961:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1802.294926] LustreError: 76961:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1802.387566] Lustre: server umount lustre-MDT0000 complete [ 1804.264849] LustreError: 73829:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1804.271302] LustreError: 73829:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 1805.261891] LustreError: 76962:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763511833 with bad export cookie 15977939306052030698 [ 1805.264700] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1805.274401] LustreError: 76962:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1805.559941] Lustre: server umount lustre-MDT0001 complete [ 1812.111581] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 19:23:58 (1763511838) [ 1817.420799] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1817.748608] LustreError: 77836:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1817.760900] LustreError: 77836:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1817.817216] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1820.941761] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1823.204510] LustreError: 77836:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1827.302298] LustreError: 77836:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1828.111622] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1831.292514] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1835.988951] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1840.194605] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1846.776568] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1852.034962] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1857.174792] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1860.164110] Lustre: 80179:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting 62c945dd-713f-49c0-a60e-f78d4686626a at adminstrative request [ 1862.378525] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1862.406365] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1872.871504] Lustre: MGS: Client 62c945dd-713f-49c0-a60e-f78d4686626a (at 0@lo) reconnecting [ 1872.875617] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xddbd0b24e3fb945c to 0xddbd0b24e3fb96fc [ 1872.880942] Lustre: MGC192.168.204.144@tcp: Connection restored to 0@lo (at 0@lo) [ 1897.404942] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1897.487077] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1897.623270] Lustre: lustre-OST0001: new disk, initializing [ 1897.629701] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1897.722357] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1897.725356] Lustre: Skipped 2 previous similar messages [ 1902.399165] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 1904.232617] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1904.246984] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1904.351418] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1909.621605] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1914.851588] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1914.857648] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1914.889831] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 1914.894587] Lustre: Skipped 5 previous similar messages [ 1918.341131] LustreError: 81313:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 1918.352837] LustreError: 81313:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 1918.418455] Lustre: server umount lustre-OST0001 complete [ 1919.973153] LustreError: 79439:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1919.985897] LustreError: 79439:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1924.064416] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1924.075254] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1924.109364] Lustre: Skipped 1 previous similar message [ 1927.917872] Lustre: server umount lustre-OST0000 complete [ 1937.399865] LustreError: 81718:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 1937.403633] LustreError: 81718:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1937.495071] Lustre: server umount lustre-MDT0000 complete [ 1940.455256] LustreError: 77831:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1940.470529] LustreError: 77831:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 1940.744522] LustreError: 77816:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763511968 with bad export cookie 15977939306052032252 [ 1940.750699] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1940.757744] LustreError: 77816:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1941.047666] Lustre: server umount lustre-MDT0001 complete [ 1948.067035] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 19:26:14 (1763511974) [ 1949.306061] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1950.606511] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 19:26:16 (1763511976) [ 1951.885707] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1953.416822] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 19:26:19 (1763511979) [ 1954.476748] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1955.853831] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 19:26:22 (1763511982) [ 1957.252058] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1961.257503] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1962.107189] loop: module loaded [ 1962.142464] loop0: detected capacity change from 0 to 4294967296 [ 1962.189912] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1963.529443] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1966.223779] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1968.909721] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1975.157433] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1978.186822] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1981.864160] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1985.915543] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1993.843739] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1993.940842] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1993.988892] LustreError: 84497:0:(obd_config.c:783:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1993.993647] LustreError: 84497:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1994.011487] LustreError: 84497:0:(tgt_mount.c:2447:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1994.030180] LustreError: 84497:0:(super25.c:199:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2011.804127] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 2018.019351] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 2025.590687] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2032.706043] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2038.614240] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2044.355898] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2050.682784] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2050.757723] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2051.005331] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2051.043740] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2051.130457] Lustre: lustre-MDT0000: new disk, initializing [ 2051.239104] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2051.251094] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2054.806552] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2064.556954] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2064.616481] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2064.759762] Lustre: 87341:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2065.005735] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2068.432184] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2074.515595] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2077.527604] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2082.959932] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2083.031764] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2087.829760] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2090.043366] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2090.135688] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2094.016498] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2097.578267] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2097.752673] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2100.442114] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2100.584503] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2105.312904] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2105.315516] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2105.327353] Lustre: Skipped 3 previous similar messages [ 2105.334995] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2105.337846] Lustre: Skipped 5 previous similar messages [ 2108.156195] LustreError: 90277:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2108.162178] LustreError: 90277:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2108.259598] Lustre: server umount lustre-OST0000 complete [ 2110.949192] LustreError: 64451:0:(client.c:1380:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000009f984293 x1849175868355456/t0(0) o8->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2125.281796] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2125.420234] Lustre: server umount lustre-MDT0000 complete [ 2128.153370] LustreError: 87332:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763512155 with bad export cookie 15977939306052032847 [ 2128.161253] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2128.168696] LustreError: 87332:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2128.452179] Lustre: server umount lustre-MDT0001 complete [ 2138.036358] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 19:29:23 (1763512163) [ 2145.680776] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2146.439340] LustreError: 91355:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2151.012868] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2159.294684] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2162.992263] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2167.873531] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2170.860028] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2176.336559] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2181.107117] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2185.281775] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2188.618234] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2193.029548] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 4 sec [ 2197.052225] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2197.243656] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2202.940630] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2203.028153] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2203.141186] Lustre: lustre-OST0001: new disk, initializing [ 2203.143180] Lustre: Skipped 2 previous similar messages [ 2203.146330] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2203.148875] Lustre: Skipped 3 previous similar messages [ 2203.223380] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2203.227298] Lustre: Skipped 5 previous similar messages [ 2208.216436] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2210.365925] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2210.379361] Lustre: Skipped 2 previous similar messages [ 2210.386128] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2210.447832] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2214.704722] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2217.473515] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2217.600597] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2220.338979] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2220.520748] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2235.576322] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2258.630508] Lustre: Failing over lustre-MDT0000 [ 2258.882527] LustreError: 96294:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 2258.885803] LustreError: 96294:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2258.944944] Lustre: server umount lustre-MDT0000 complete [ 2261.984611] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2261.986161] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2261.989536] LustreError: 92417:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2261.989547] LustreError: 92417:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2261.994919] LustreError: Skipped 2 previous similar messages [ 2261.999848] Lustre: Skipped 4 previous similar messages [ 2277.343306] Lustre: 64453:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1763512289/real 1763512289] req@000000004e895038 x1849175868423040/t0(0) o400->MGC192.168.204.144@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1763512305 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2277.367543] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2280.348256] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2280.350161] LDISKFS-fs (dm-0): recovery complete [ 2280.357896] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2287.596124] LustreError: 64451:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000b09f81af x1849175868431104/t0(0) o250->MGC192.168.204.144@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2287.824364] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2289.943273] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2289.944646] Lustre: 91352:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2290.244644] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2293.220817] Lustre: 91355:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2293.230463] Lustre: 91355:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 1 previous similar message [ 2293.243378] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 2293.285814] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2293.607214] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2293.934083] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2293.939548] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 1 previous similar message [ 2294.255175] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2294.262089] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 1 previous similar message [ 2294.275370] Lustre: 97043:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2294.290301] Lustre: 97043:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 7 previous similar messages [ 2295.243943] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2295.251347] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 3 previous similar messages [ 2295.568408] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2295.576360] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 3 previous similar messages [ 2296.544132] Lustre: 97043:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2296.554287] Lustre: 97043:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 6 previous similar messages [ 2297.507188] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2297.513111] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 6 previous similar messages [ 2297.831692] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2297.836146] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 6 previous similar messages [ 2300.674070] Lustre: 97043:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2300.682485] Lustre: 97043:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 12 previous similar messages [ 2301.653973] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2301.665493] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 12 previous similar messages [ 2301.983044] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2301.991449] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 12 previous similar messages [ 2308.920146] Lustre: 97043:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2308.930212] Lustre: 97043:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 25 previous similar messages [ 2309.882196] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2309.895421] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 25 previous similar messages [ 2310.215675] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2310.220638] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 25 previous similar messages [ 2325.082335] Lustre: 97043:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2325.094822] Lustre: 97043:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 50 previous similar messages [ 2326.048874] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2326.060320] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 50 previous similar messages [ 2326.383083] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2326.392433] LustreError: 97043:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 50 previous similar messages [ 2349.183146] Lustre: 97043:0:(ldlm_lib.c:2375:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2349.189625] Lustre: 97043:0:(ldlm_lib.c:2385:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2349.196694] Lustre: 97043:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 743e8f0d-e699-4035-b455-ed7b6b0568d8@192.168.204.44@tcp [ 2349.210246] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2349.216525] Lustre: 97043:0:(ldlm_lib.c:1897:abort_req_replay_queue()) @@@ aborted: req@00000000dfe74e3d x1849175854615424/t0(8589934773) o101->743e8f0d-e699-4035-b455-ed7b6b0568d8@192.168.204.44@tcp:752/0 lens 592/0 e 0 to 0 dl 1763512387 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2349.234862] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2349.252151] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2349.260910] Lustre: lustre-MDT0000: Denying connection for new client 743e8f0d-e699-4035-b455-ed7b6b0568d8 (at 192.168.204.44@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) already passed deadline 0:00 [ 2349.327347] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2349.340738] Lustre: Skipped 2 previous similar messages [ 2349.342920] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2349.445958] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2349.448110] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 2364.388600] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2364.401388] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2364.414585] Lustre: Skipped 1 previous similar message [ 2364.421372] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2364.424350] Lustre: Skipped 9 previous similar messages [ 2368.800983] Lustre: server umount lustre-OST0000 complete [ 2370.033676] LustreError: 92953:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2370.058178] LustreError: 92953:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 36 previous similar messages [ 2378.512830] Lustre: server umount lustre-OST0001 complete [ 2388.698322] LustreError: 98106:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 2388.701439] LustreError: 98106:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 2388.814814] Lustre: server umount lustre-MDT0000 complete [ 2391.217368] LustreError: 91337:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763512419 with bad export cookie 15977939306052084059 [ 2391.220856] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2391.222928] LustreError: 91337:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2391.401118] Lustre: server umount lustre-MDT0001 complete [ 2397.493269] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 19:33:43 (1763512423) [ 2403.047745] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2403.053890] Lustre: *** cfs_fail_loc=197, val=0*** [ 2418.527590] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2423.648297] LustreError: 98999:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.144@tcp: failed processing log, type 4: rc = -110 [ 2428.812252] Lustre: *** cfs_fail_loc=197, val=0*** [ 2428.815981] Lustre: Skipped 65 previous similar messages [ 2453.427766] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2457.479251] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2459.010762] Lustre: server umount lustre-OST0000 complete [ 2466.117449] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 19:34:52 (1763512492) [ 2484.712336] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2485.121196] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2485.124538] Lustre: Skipped 2 previous similar messages [ 2487.941124] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2493.409091] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2496.872455] LustreError: 100538:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2496.883373] LustreError: 100538:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2496.899985] LustreError: 100538:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 12, retries 0, failed: rc = -5 [ 2497.325985] Lustre: server umount lustre-MDT0000 complete [ 2513.755795] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2526.913312] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 2533.843458] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 2540.961182] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2545.829309] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2550.567543] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2555.260790] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2560.350364] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2560.427581] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2560.613158] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2560.653044] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2560.761656] Lustre: lustre-MDT0000: new disk, initializing [ 2560.872408] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2564.061806] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2573.200135] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2573.272333] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2573.396414] Lustre: 104577:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2573.613231] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2577.130630] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2583.681462] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2586.598724] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2592.079270] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2592.127475] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2596.975310] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2600.584490] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2602.862879] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2606.379847] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2606.536168] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2609.369978] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2609.550296] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2611.177182] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2611.185388] Lustre: Skipped 5 previous similar messages [ 2611.192036] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2611.195125] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2611.196601] LustreError: Skipped 2 previous similar messages [ 2611.199187] Lustre: Skipped 6 previous similar messages [ 2617.247426] Lustre: server umount lustre-OST0000 complete [ 2626.269598] Lustre: server umount lustre-MDT0000 complete [ 2626.531306] LustreError: 104589:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2626.549413] LustreError: 104589:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [ 2628.712830] LustreError: 104571:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763512656 with bad export cookie 15977939306052094657 [ 2628.715210] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2628.720170] LustreError: 104571:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2628.917637] Lustre: server umount lustre-MDT0001 complete [ 2635.023198] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 19:37:41 (1763512661) [ 2641.764768] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 2643.996739] Key type lgssc unregistered [ 2644.263831] LNet: 108647:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2644.272171] LNetError: 108647:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2644.284936] LNet: Removed LNI 192.168.204.144@tcp [ 2644.979201] Key type .llcrypt unregistered [ 2644.981409] Key type ._llcrypt unregistered [ 2660.703996] Key type ._llcrypt registered [ 2660.705689] Key type .llcrypt registered [ 2660.824864] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 2672.421163] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 2673.083815] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2673.108671] alg: No test for adler32 (adler32-zlib) [ 2674.089597] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 2674.273341] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 2675.903269] Key type lgssc registered [ 2677.182212] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2685.801648] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2691.238485] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2697.403261] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2703.125501] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2707.737752] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2712.686703] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2712.755505] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2712.765924] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2713.935809] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2713.962462] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2714.013823] Lustre: lustre-MDT0000: new disk, initializing [ 2714.099368] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2714.116752] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2716.787316] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2724.656239] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2724.705834] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2724.860081] Lustre: lustre-OST0000: new disk, initializing [ 2724.863427] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2724.868722] Lustre: Skipped 1 previous similar message [ 2724.929370] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2726.789874] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2726.798442] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2726.823687] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2729.323125] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2737.565203] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2737.614737] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2737.737476] Lustre: lustre-OST0001: new disk, initializing [ 2737.746222] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2737.816808] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2742.344589] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2747.486149] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2747.497361] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2747.540332] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2752.484089] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2752.502283] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2756.482321] LustreError: 114205:0:(obd_class.h:479:obd_check_dev()) Device 17 not setup [ 2756.666547] Lustre: server umount lustre-MDT0000 complete [ 2762.079945] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2766.524534] LustreError: 113613:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763512794 with bad export cookie 13358281343867090325 [ 2766.527135] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2766.575526] LustreError: 113613:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2766.620094] LustreError: 114800:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 2766.632428] LustreError: 114800:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2766.729480] Lustre: server umount lustre-OST0000 complete [ 2769.667529] LustreError: 115001:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 2769.675873] LustreError: 115001:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 2769.797336] Lustre: server umount lustre-OST0001 complete [ 2776.717533] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 19:40:02 (1763512802) [ 2783.669574] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 2790.403511] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2791.222636] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2795.853675] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2796.662423] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2801.872202] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 19:40:28 (1763512828) [ 2812.657913] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2814.124030] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2815.624958] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2817.149300] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2817.833849] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2819.501230] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2831.543693] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 2837.828117] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 2845.128850] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2850.316946] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2855.142470] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2860.888954] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2867.957613] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2868.013605] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2868.371252] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2868.417818] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2868.547807] Lustre: lustre-MDT0000: new disk, initializing [ 2868.704897] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2868.759125] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2872.981597] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2881.265738] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2881.310499] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2881.380481] Lustre: 120329:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2881.421317] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2881.424310] Lustre: Skipped 1 previous similar message [ 2881.502571] Lustre: lustre-MDT0001: new disk, initializing [ 2881.580633] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2881.632071] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2881.639699] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2884.288772] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2890.418277] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2893.489247] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2899.762700] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2899.807841] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2900.060953] Lustre: lustre-OST0000: new disk, initializing [ 2900.079670] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2900.196447] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2904.741891] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2910.257519] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2910.275695] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2910.393551] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2911.607381] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2915.589743] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2915.826725] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2919.432790] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2919.624390] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2926.049965] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2926.056300] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2926.075165] LustreError: Skipped 1 previous similar message [ 2926.079330] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2926.079338] Lustre: Skipped 1 previous similar message [ 2926.094285] Lustre: Skipped 2 previous similar messages [ 2927.431102] LustreError: 123266:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2927.444346] LustreError: 123266:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2927.633756] Lustre: server umount lustre-OST0000 complete [ 2931.170919] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2931.201624] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2931.204116] Lustre: Skipped 2 previous similar messages [ 2936.288114] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2936.290636] Lustre: Skipped 1 previous similar message [ 2937.139659] LustreError: 123468:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2937.146867] LustreError: 123468:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2937.291477] Lustre: server umount lustre-MDT0000 complete [ 2940.583188] LustreError: 120322:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763512968 with bad export cookie 13358281343867091550 [ 2940.590688] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2940.862321] Lustre: server umount lustre-MDT0001 complete [ 2948.152944] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 19:42:54 (1763512974) [ 2953.722620] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2954.114429] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2957.517482] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2959.332376] LustreError: 124343:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2959.351958] LustreError: 124343:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2963.426818] LustreError: 124343:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2964.437168] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2967.351749] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2971.567951] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2974.986161] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2980.437262] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2984.606815] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 2989.091534] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3003.417859] Lustre: *** cfs_fail_loc=159, val=0*** [ 3003.427681] Lustre: Skipped 3 previous similar messages [ 3019.054129] Lustre: lustre-MDT0000: Client df2037cb-ddc0-4219-8efe-1242d2ce95b6 (at 192.168.204.44@tcp) reconnecting [ 3020.463503] Lustre: *** cfs_fail_loc=159, val=0*** [ 3020.464904] Lustre: Skipped 2 previous similar messages [ 3036.032723] Lustre: lustre-MDT0000: Client df2037cb-ddc0-4219-8efe-1242d2ce95b6 (at 192.168.204.44@tcp) reconnecting [ 3041.249374] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3041.252957] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3041.258504] Lustre: Skipped 1 previous similar message [ 3041.265787] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3042.499106] LustreError: 126854:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3042.501550] LustreError: 126854:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 3042.624160] Lustre: server umount lustre-OST0000 complete [ 3046.880516] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3046.886797] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3050.468129] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3050.471237] Lustre: Skipped 1 previous similar message [ 3055.659365] Lustre: server umount lustre-MDT0000 complete [ 3058.747370] LustreError: 124322:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763513086 with bad export cookie 13358281343867092432 [ 3058.754432] LustreError: 124322:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3058.755607] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3058.872844] LustreError: 127257:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3058.875457] LustreError: 127257:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 3059.039492] Lustre: server umount lustre-MDT0001 complete [ 3068.189722] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 3070.543481] Key type lgssc unregistered [ 3070.767880] LNet: 127789:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3070.772781] LNetError: 127789:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3070.795710] LNet: Removed LNI 192.168.204.144@tcp [ 3071.527464] Key type .llcrypt unregistered [ 3071.529144] Key type ._llcrypt unregistered [ 3094.401612] Key type ._llcrypt registered [ 3094.405456] Key type .llcrypt registered [ 3094.514615] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 3095.482776] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3095.490026] alg: No test for adler32 (adler32-zlib) [ 3096.550355] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 3096.788397] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 3098.471222] Key type lgssc registered [ 3099.501944] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3105.636828] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3105.646725] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3107.053206] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3110.380498] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3116.520879] LustreError: 129183:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3116.541529] LustreError: 129183:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3117.984485] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3118.278679] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3121.695782] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3126.614964] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3129.955724] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3135.171652] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3135.512027] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3140.382939] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3144.600951] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3151.920202] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3159.247609] Lustre: *** cfs_fail_loc=159, val=0*** [ 3175.753244] Lustre: lustre-MDT0000: Client 3f7f43b9-3ec3-4a27-9a8e-8aa53c41254d (at 192.168.204.44@tcp) reconnecting [ 3177.440627] Lustre: *** cfs_fail_loc=159, val=0*** [ 3193.674612] Lustre: lustre-MDT0001: Client 3f7f43b9-3ec3-4a27-9a8e-8aa53c41254d (at 192.168.204.44@tcp) reconnecting [ 3194.836118] Lustre: *** cfs_fail_loc=159, val=0*** [ 3194.844208] Lustre: Skipped 4 previous similar messages [ 3211.070056] Lustre: lustre-MDT0001: Client 3f7f43b9-3ec3-4a27-9a8e-8aa53c41254d (at 192.168.204.44@tcp) reconnecting [ 3214.012957] Lustre: 131903:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3216.397565] Lustre: *** cfs_fail_loc=159, val=0*** [ 3216.408173] Lustre: Skipped 4 previous similar messages [ 3232.084853] Lustre: lustre-MDT0000: Client 58bb4549-16f6-47ee-a9e3-d4551a8ac515 (at 192.168.204.44@tcp) reconnecting [ 3234.090350] Lustre: *** cfs_fail_loc=159, val=0*** [ 3234.092362] Lustre: Skipped 11 previous similar messages [ 3250.492616] Lustre: lustre-MDT0000: Client 58bb4549-16f6-47ee-a9e3-d4551a8ac515 (at 192.168.204.44@tcp) reconnecting [ 3251.654525] Lustre: 132189:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3255.782706] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 3255.789421] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3255.801488] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3256.288695] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3256.295594] LustreError: Skipped 5 previous similar messages [ 3256.299540] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3256.306964] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3257.535813] LustreError: 132288:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3257.641318] Lustre: server umount lustre-OST0000 complete [ 3261.922318] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3261.927543] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3261.933933] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3264.481959] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3264.484162] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3266.572911] LustreError: 132488:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3266.576234] LustreError: 132488:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3266.633334] Lustre: server umount lustre-MDT0000 complete [ 3268.981793] LustreError: 129902:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763513296 with bad export cookie 4480548427104403522 [ 3268.984217] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3268.987606] LustreError: 129902:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3269.092859] LustreError: 132690:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3269.097482] LustreError: 132690:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3269.216870] Lustre: server umount lustre-MDT0001 complete [ 3275.730388] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 3277.645791] Key type lgssc unregistered [ 3277.864348] LNet: 133222:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3277.874403] LNetError: 133222:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3277.893281] LNet: Removed LNI 192.168.204.144@tcp [ 3278.474303] Key type .llcrypt unregistered [ 3278.476555] Key type ._llcrypt unregistered [ 3294.060911] Key type ._llcrypt registered [ 3294.062451] Key type .llcrypt registered [ 3294.150195] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 3295.000766] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3295.026067] alg: No test for adler32 (adler32-zlib) [ 3296.072659] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 3296.242414] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 3297.903181] Key type lgssc registered [ 3298.683482] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3304.336362] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3304.352958] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3305.650911] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3308.686678] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3314.420519] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3314.693296] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3317.415275] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3321.195621] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3324.117330] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3329.520553] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3329.777088] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3330.804650] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3334.008278] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3338.673906] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3340.042080] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3343.420295] Lustre: 136849:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3345.191982] Lustre: 136897:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3346.413834] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3346.939919] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3349.129751] Lustre: 137045:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3349.675326] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3350.142472] Lustre: 137143:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3354.170323] LustreError: 137240:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3354.272574] Lustre: server umount lustre-OST0000 complete [ 3360.736988] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3360.738401] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3360.749668] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3365.856543] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3365.858423] Lustre: Skipped 1 previous similar message [ 3366.886953] LustreError: 137448:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3366.889720] LustreError: 137448:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3366.939266] Lustre: server umount lustre-MDT0000 complete [ 3369.526395] LustreError: 135316:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763513397 with bad export cookie 14128370188436989074 [ 3369.531810] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3369.539941] LustreError: 135316:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3379.690718] LustreError: 137650:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3379.697744] LustreError: 137650:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3379.836394] Lustre: server umount lustre-MDT0001 complete [ 3386.849631] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 3388.869489] Key type lgssc unregistered [ 3389.078681] LNet: 138182:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3389.089382] LNetError: 138182:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3389.102547] LNet: Removed LNI 192.168.204.144@tcp [ 3389.776470] Key type .llcrypt unregistered [ 3389.778915] Key type ._llcrypt unregistered [ 3409.288397] Key type ._llcrypt registered [ 3409.290491] Key type .llcrypt registered [ 3409.357956] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 3410.136733] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3410.154299] alg: No test for adler32 (adler32-zlib) [ 3411.127601] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 3411.306437] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 3412.935175] Key type lgssc registered [ 3413.919341] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3419.801818] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3419.821298] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3421.121864] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3423.552550] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3428.727572] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3429.094893] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3431.509235] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3435.831710] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3438.475571] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3443.774758] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3444.091517] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3446.128880] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 3447.699583] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3451.094966] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3453.309560] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 3459.345880] Lustre: *** cfs_fail_loc=159, val=0*** [ 3475.760647] Lustre: lustre-MDT0000: Client 9f9dad5c-f5b4-4e44-92e5-a40c0bc22adf (at 192.168.204.44@tcp) reconnecting [ 3479.011274] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3479.012760] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3479.013208] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3479.015859] Lustre: Skipped 1 previous similar message [ 3484.128710] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3484.131543] Lustre: Skipped 1 previous similar message [ 3484.471364] LustreError: 141951:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3484.578292] Lustre: server umount lustre-OST0000 complete [ 3489.255867] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3489.264098] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3489.264206] Lustre: Skipped 1 previous similar message [ 3493.847780] LustreError: 142152:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3493.873888] LustreError: 142152:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3493.923290] Lustre: server umount lustre-MDT0000 complete [ 3494.371586] LustreError: 140290:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3494.381984] LustreError: 140290:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3496.430719] LustreError: 139554:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763513524 with bad export cookie 16666351611971633584 [ 3496.435985] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3496.436404] LustreError: 139554:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3496.570987] LustreError: 142354:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3496.573597] LustreError: 142354:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3496.667711] Lustre: server umount lustre-MDT0001 complete [ 3505.286683] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 3507.863290] Key type lgssc unregistered [ 3508.149221] LNet: 142886:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3508.158798] LNetError: 142886:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3508.175835] LNet: Removed LNI 192.168.204.144@tcp [ 3508.912697] Key type .llcrypt unregistered [ 3508.914498] Key type ._llcrypt unregistered [ 3530.602097] Key type ._llcrypt registered [ 3530.603450] Key type .llcrypt registered [ 3530.658855] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 3531.088797] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3531.105722] alg: No test for adler32 (adler32-zlib) [ 3532.092383] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 3532.216039] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 3533.831701] Key type lgssc registered [ 3534.590496] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3540.101385] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3540.122787] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3541.534338] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3544.564553] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3551.165515] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3551.458285] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3554.440648] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3558.666734] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3561.912831] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3567.627480] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3567.780798] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 3567.858850] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3571.714320] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3573.279372] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 3576.307150] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 3576.488446] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3593.876674] Lustre: lustre-OST0000: evicting fa671cab-6f43-407e-bff9-fee608c7a783 (at 192.168.204.44@tcp) by administrative request [ 3593.881922] Lustre: Skipped 1 previous similar message [ 3622.372494] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3622.378864] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3622.381064] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3622.382654] Lustre: Skipped 1 previous similar message [ 3626.689872] LustreError: 147065:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3626.783759] Lustre: server umount lustre-OST0000 complete [ 3632.612581] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3632.618402] Lustre: Skipped 1 previous similar message [ 3632.621064] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3632.631607] Lustre: Skipped 1 previous similar message [ 3635.691281] LustreError: 147266:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3635.693517] LustreError: 147266:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3635.768199] Lustre: server umount lustre-MDT0000 complete [ 3637.731450] LustreError: 144278:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3637.741818] LustreError: 144278:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3638.813001] LustreError: 144264:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763513666 with bad export cookie 17720136279004083109 [ 3638.819896] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3638.829873] LustreError: 144264:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3638.959347] LustreError: 147468:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3638.961915] LustreError: 147468:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3639.188380] Lustre: server umount lustre-MDT0001 complete [ 3647.363922] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 3649.671081] Key type lgssc unregistered [ 3649.917619] LNet: 148000:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3649.930193] LNetError: 148000:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3649.943625] LNet: Removed LNI 192.168.204.144@tcp [ 3650.739285] Key type .llcrypt unregistered [ 3650.741817] Key type ._llcrypt unregistered [ 3711.626814] Key type ._llcrypt registered [ 3711.632200] Key type .llcrypt registered [ 3711.696711] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 3712.196971] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3712.205541] alg: No test for adler32 (adler32-zlib) [ 3713.202390] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 3713.371583] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 3715.031329] Key type lgssc registered [ 3715.790702] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3721.290801] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3721.305749] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3722.685717] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3725.555707] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3731.852422] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3732.165090] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3735.266854] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3739.147708] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3741.745040] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3746.959154] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3747.213214] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3748.277025] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 3751.719454] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3756.128549] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3756.495879] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 3763.686806] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3763.690180] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3763.704941] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3766.754628] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3766.756768] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3766.772857] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3769.600373] LustreError: 152723:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3769.696876] Lustre: server umount lustre-OST0000 complete [ 3773.420935] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3773.424538] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3773.435715] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3773.438158] Lustre: Skipped 1 previous similar message [ 3776.994078] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3776.994616] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3778.541293] LustreError: 152925:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3778.557832] LustreError: 152925:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3778.648202] Lustre: server umount lustre-MDT0000 complete [ 3781.949700] LustreError: 150449:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763513809 with bad export cookie 5620949384403158224 [ 3781.951790] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3781.960283] LustreError: 150449:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3782.091873] LustreError: 153127:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3782.094824] LustreError: 153127:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3782.248843] Lustre: server umount lustre-MDT0001 complete [ 3793.314017] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 3795.785822] Key type lgssc unregistered [ 3796.095173] LNet: 153659:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3796.104169] LNetError: 153659:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3796.128938] LNet: Removed LNI 192.168.204.144@tcp [ 3797.006484] Key type .llcrypt unregistered [ 3797.009467] Key type ._llcrypt unregistered [ 3815.791274] Key type ._llcrypt registered [ 3815.792919] Key type .llcrypt registered [ 3815.877325] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 3816.773533] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3816.780729] alg: No test for adler32 (adler32-zlib) [ 3817.739100] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 3817.877375] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 3819.495232] Key type lgssc registered [ 3820.336270] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3825.077207] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3830.284723] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 19:57:36 (1763513856) [ 3843.142766] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 3851.296374] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 3858.752711] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3864.707574] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3869.656253] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3874.137357] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3879.354457] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3879.398441] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3879.407900] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3880.589899] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3880.627341] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3880.729539] Lustre: lustre-MDT0000: new disk, initializing [ 3880.825654] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3880.835631] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3884.632039] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3893.092912] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3893.187927] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3893.271703] Lustre: 157860:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3893.324368] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3893.328370] Lustre: Skipped 1 previous similar message [ 3893.423884] Lustre: lustre-MDT0001: new disk, initializing [ 3893.497368] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3893.538943] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3893.549204] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3896.676951] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3902.732737] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3905.850474] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3910.818500] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3910.877358] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3911.078818] Lustre: lustre-OST0000: new disk, initializing [ 3911.083984] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3911.144520] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3915.149862] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 3918.880100] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3918.893137] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3918.968361] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3920.324588] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3929.058937] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3929.062176] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3929.066210] Lustre: Skipped 1 previous similar message [ 3929.066652] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3929.069575] Lustre: Skipped 1 previous similar message [ 3933.501201] LustreError: 160444:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3933.612184] Lustre: server umount lustre-OST0000 complete [ 3939.301309] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3939.304173] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3939.319654] Lustre: Skipped 1 previous similar message [ 3943.079211] LustreError: 160646:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 3943.082310] LustreError: 160646:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3943.161535] Lustre: server umount lustre-MDT0000 complete [ 3946.084856] LustreError: 157851:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763513973 with bad export cookie 3740148434048495155 [ 3946.094181] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3946.196295] LustreError: 160847:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3946.208181] LustreError: 160847:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3946.489864] Lustre: server umount lustre-MDT0001 complete [ 3955.549455] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 3957.313126] Key type lgssc unregistered [ 3957.506615] LNet: 161380:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3957.511442] LNetError: 161380:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3957.525588] LNet: Removed LNI 192.168.204.144@tcp [ 3958.055606] Key type .llcrypt unregistered [ 3958.057672] Key type ._llcrypt unregistered [ 3978.301634] Key type ._llcrypt registered [ 3978.303337] Key type .llcrypt registered [ 3978.453384] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 3979.126120] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3979.220067] alg: No test for adler32 (adler32-zlib) [ 3980.239510] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 3980.432822] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 3982.071499] Key type lgssc registered [ 3983.040403] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3989.192933] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3989.222260] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3990.812053] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3994.196355] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 4000.231572] LustreError: 162773:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4000.238606] LustreError: 162773:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4001.608275] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4002.038176] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4005.109423] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 4010.300819] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4013.559090] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4019.031442] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4019.453767] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4024.381478] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 4028.756703] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4035.834450] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4043.782664] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4043.822167] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4043.845219] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4043.853970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4043.868310] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4044.584612] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4044.595444] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4044.612732] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4044.626829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4044.642260] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4045.379990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4046.001499] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4046.004603] Lustre: Skipped 1 previous similar message [ 4046.018994] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4046.025471] Lustre: Skipped 1 previous similar message [ 4046.027737] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4046.030989] Lustre: Skipped 1 previous similar message [ 4046.035496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4046.041582] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4046.046763] Lustre: Skipped 1 previous similar message [ 4046.697832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4047.331916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4047.972278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4048.526903] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4048.530376] Lustre: Skipped 3 previous similar messages [ 4048.540595] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4048.553071] Lustre: Skipped 3 previous similar messages [ 4048.558750] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4048.565187] Lustre: Skipped 3 previous similar messages [ 4048.569028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4048.577734] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4048.580364] Lustre: Skipped 3 previous similar messages [ 4049.219262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4049.886926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4050.610870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4051.235867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4051.876789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4052.714283] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4052.719483] Lustre: Skipped 5 previous similar messages [ 4052.729119] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4052.740449] Lustre: Skipped 5 previous similar messages [ 4052.743867] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4052.747312] Lustre: Skipped 5 previous similar messages [ 4052.753565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4052.759101] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4052.761363] Lustre: Skipped 5 previous similar messages [ 4053.658114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4054.466108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4055.255361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4056.059967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4056.781255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4057.459870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4058.188577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4058.922399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4059.605555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4060.254784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4060.824049] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4060.827359] Lustre: Skipped 10 previous similar messages [ 4060.835772] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4060.841852] Lustre: Skipped 10 previous similar messages [ 4060.845386] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4060.848176] Lustre: Skipped 10 previous similar messages [ 4060.855806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4060.868823] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4060.871691] Lustre: Skipped 10 previous similar messages [ 4061.611670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4062.336580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4063.072194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4063.854420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4064.594804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4065.317528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4066.163448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4066.964508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4067.652677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4068.195420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4069.052198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4069.772119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4070.728517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4071.532293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4072.275834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4072.977386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4073.863400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4074.737973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4075.723461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4076.633520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4077.401805] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4077.407714] Lustre: Skipped 20 previous similar messages [ 4077.446082] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4077.463228] Lustre: Skipped 20 previous similar messages [ 4077.466825] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4077.469932] Lustre: Skipped 20 previous similar messages [ 4077.474831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4077.480220] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4077.482566] Lustre: Skipped 20 previous similar messages [ 4078.239544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4078.958469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4079.712498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4080.499288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4081.319842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4082.126601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4082.923079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4083.866724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4084.727887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4085.561354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4086.429896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4087.386861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4088.275521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4089.123162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4090.042256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4090.946807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4091.746215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4092.512799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4093.272732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4094.077848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4094.803226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4095.490398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4096.168245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4096.836085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4097.578566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4098.388021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4099.074451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4099.876652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4100.662242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4101.492984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4102.267360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4103.212886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4103.958316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4104.914949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4105.738173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4106.546538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4107.526433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4108.510161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4109.422827] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4109.431721] Lustre: Skipped 38 previous similar messages [ 4109.448171] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4109.453878] Lustre: Skipped 38 previous similar messages [ 4109.456416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4110.453486] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4110.461811] Lustre: Skipped 39 previous similar messages [ 4110.466635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4110.477253] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4110.481890] Lustre: Skipped 39 previous similar messages [ 4111.426823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4112.392603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4113.305829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4114.188628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4115.350229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4116.015876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4116.784671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4117.520207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4118.284417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4119.041681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4119.878368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4120.633977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4121.403696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4122.272958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4123.252337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4124.074909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4124.946129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4125.706155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4126.495291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4127.270591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4128.051952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4128.829331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4129.627534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4130.349774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4131.101656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4131.962445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4132.735735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4133.537468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4134.374197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4135.297363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4135.946884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4136.694863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4137.504044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4138.275450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4139.198849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4140.215402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.219176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4142.235265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4143.270454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4144.032227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4144.728228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4145.446649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4146.334805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.096980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.732081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.350510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.984861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4149.613537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4150.267628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4151.005268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4151.817561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.654815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.497925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4154.403807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4155.249562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4155.998867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4156.759494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4157.508442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.217626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.996771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4159.683357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4160.467735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4161.201327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4161.968663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4162.796985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4163.518810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4164.333359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4165.113823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4165.951767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4166.690640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4167.669843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4168.442791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4169.306040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4170.142029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.066510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.866476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4172.624904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4173.338540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.036610] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4174.044799] Lustre: Skipped 79 previous similar messages [ 4174.059609] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4174.075414] Lustre: Skipped 79 previous similar messages [ 4174.082880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.833363] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4174.836907] Lustre: Skipped 79 previous similar messages [ 4174.840596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.850851] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4174.853770] Lustre: Skipped 79 previous similar messages [ 4175.594701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4176.373424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.126732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.885293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4178.711352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4179.678341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.607163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.446700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.423875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4183.234236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4183.978195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4184.616894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.756769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4186.591944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4187.813926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4188.577178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4189.488100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.323505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.174157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4192.112739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4193.270756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4194.168643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4195.268919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4196.102703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4197.303458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4198.462522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4199.551392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4200.528470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4201.720670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4202.752249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.050216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4205.144237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4206.214409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4207.200340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4208.235355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4209.182868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4210.089466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4210.945277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4211.785614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4212.555619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4213.502467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4214.327558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4215.180301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4216.079854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.209181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4218.024669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4218.960292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4219.933502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4220.810409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4221.729225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.511569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4223.443135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4224.304486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4225.209196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4226.027577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4226.945717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4227.769065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4228.599960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4229.607244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4230.511094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4231.401811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4232.275530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4233.119789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4234.051432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4235.056721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4235.837521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4236.728993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4237.504690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4238.284394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4239.005393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4239.796624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4240.540745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4241.328729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4242.077206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4243.069593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4243.786803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4244.495845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4245.130632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4245.787635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4246.485919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4247.309349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4248.082155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4248.782441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4249.599272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4250.321425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4251.091145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4251.917662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4252.782735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4253.676682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4254.601025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4255.447374] LustreError: 162793:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4255.453659] LustreError: 162793:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4255.467798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4256.402857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4257.572297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4258.432335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4259.156536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4260.124153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4261.022849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4261.868213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4262.648783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4263.455784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4264.465287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4265.159510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4265.953978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4266.831735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4267.746479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4268.560051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4269.311726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4270.012168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4270.762243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4271.536079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4272.333343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4272.988948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4273.634571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4274.352885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4275.264519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4276.052868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4276.951518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4277.559986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4278.244045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4278.914411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4279.576520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4280.344775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4281.183272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4281.884583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4282.915303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4283.747662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4284.649945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4285.458153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4286.434687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4287.266658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4288.197038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4289.178994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4290.197664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4290.894852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4291.721826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4292.643066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4293.557952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4294.508180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4295.571951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4296.530875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4297.505792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4298.409261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4299.494551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4300.515877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4301.624127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4302.553868] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4302.579929] Lustre: Skipped 146 previous similar messages [ 4302.592671] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4302.608202] Lustre: Skipped 146 previous similar messages [ 4302.614948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4303.634499] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4303.641560] Lustre: Skipped 146 previous similar messages [ 4303.645328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4303.652065] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4303.654832] Lustre: Skipped 146 previous similar messages [ 4304.443680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4305.211118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4306.024930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4306.711357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4307.390220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4308.054617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4308.670724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4309.287215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4309.936742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4310.577457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4311.211776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4311.772494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4312.308929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4312.905946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4313.699647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4314.561898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4315.281218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4316.158772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4316.888623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4317.643303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4318.375589] LustreError: 162793:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4318.385376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4318.391074] LustreError: 162793:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4319.175372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4319.942866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4320.653100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4321.553162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4322.436196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4323.372798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4324.275675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4325.054574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4325.844668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4326.736138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4327.691506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4328.679992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4329.516766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4330.335804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4331.052959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4331.859652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4332.563764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4333.380678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4334.236539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4335.153384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4336.137657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4337.003786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4338.017282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4338.872720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4339.729338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4340.549909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4341.395691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4342.230298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4343.196063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4344.175216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4345.197488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4346.082065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4347.129968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4348.076346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4348.952710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4349.740645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4350.516503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4351.429346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4352.164935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4352.908785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4353.749409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4354.430495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4355.137891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4356.004318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4356.796453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4357.521725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4358.476470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4359.263292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4360.130150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4360.897119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4361.744332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4362.508990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4363.315480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4364.079792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4364.885030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4365.721459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4366.729470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4367.726269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4368.622888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4369.329043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4370.077167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4370.901295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4371.651291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4372.406446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4373.179142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4374.104092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4374.925906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4375.765607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4376.387209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4377.168307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4377.836728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4378.683383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4379.409390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4380.228215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4380.962731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4381.595740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4382.361739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4383.092506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4383.899870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4384.642455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4385.448183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4386.173038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4387.128700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4387.920155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4388.749774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4389.688891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4390.697435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4391.447337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4392.172161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4392.880127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4393.659575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4394.330620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4395.040779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4395.699761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4396.285320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4397.001367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4397.599301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4398.252995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4398.824783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4399.570867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4400.238514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4401.010485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4401.743950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4402.381532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4402.993153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4403.648445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4404.384282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4405.080899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4405.843727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4406.459552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4407.022796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4407.629895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4408.273589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4409.023894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4409.780175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4410.444282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4411.198275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4411.823097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4412.402837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4413.003590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4413.686799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4414.434437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4415.143971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4415.862954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4416.576357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4417.176446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4417.778719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4418.388726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4418.953622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4419.652423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4420.401309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4421.070142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4421.678211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4422.317109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4422.959060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4423.661702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4424.361696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4425.049204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4425.801740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4426.563519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4427.468630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4428.235209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4429.034081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4429.839823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4430.729819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4431.599313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4432.445499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4433.114175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4433.810180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4434.556422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4435.200753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4435.866347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.535836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4437.131837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4437.786320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4438.492241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4439.150957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4439.757390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4440.525304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4441.238399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4441.925884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4442.649881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4443.357085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4444.004237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4444.841063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4445.619092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4446.472400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4447.145370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4447.948158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4448.623778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4449.417761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4450.141300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4450.883277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4451.580775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4452.309316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4453.169936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4453.917407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4454.629312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4455.368756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4456.118187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4456.883088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4457.532626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4458.137127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4458.771750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4459.451931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4460.036265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4460.717378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4461.395367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4462.052507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4462.703721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4463.259750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4463.860562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4464.541995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4465.063895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4465.775788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4466.486760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4467.119976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4467.833422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4468.363748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4469.045960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4469.692235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4470.394300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4471.141814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4471.908305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4472.569982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4473.410701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4474.136913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4474.854582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4475.638845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4476.407608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4477.134838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4477.932404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4478.610845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4479.319417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4480.004552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4480.703945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4481.403529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4482.189736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4482.794307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4483.425867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4484.087844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4484.766713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4485.395256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4486.152368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4486.849410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4487.507894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4488.278075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4489.042114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4489.782232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4490.598862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4491.444272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4492.172634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4492.872467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4493.625855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4494.425247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4495.184515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4496.078636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4496.958293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4497.876842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4498.615684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4499.392390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4500.073363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4500.898086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4501.718144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4502.562467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4503.251245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4503.959583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4504.643835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4505.332927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4505.945464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4506.528455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4507.118510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4507.704386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4508.337855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4509.046222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4509.682084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4510.354880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4511.039585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4511.754950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4512.459276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4513.069286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4513.915648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4514.619069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4515.375220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4516.086845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4516.933488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4517.820227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4518.822600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4519.581898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4520.508769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4521.342935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4522.144514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4522.922717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4523.643649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4524.415569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4525.193926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4525.995410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4526.682748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4527.483422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4528.226185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4529.057283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4529.777293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4530.615628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4531.358574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4532.067466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4532.875182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4533.670266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4534.346556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4535.114398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4535.815597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4536.503647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4537.224939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4537.904892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4538.538776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4539.180239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4539.821641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4540.595769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4541.169666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4541.955479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4542.767684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4543.391188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4544.003715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4544.645696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4545.274243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4545.948235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4546.563362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4547.233549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4548.086993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4548.710491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4549.540742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4550.251901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4551.014598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4551.653152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4552.300744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4553.016960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4553.636357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4554.356153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4555.078571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4555.824160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4556.514862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4557.077958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4557.574546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4558.201377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4558.925754] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4558.935167] Lustre: Skipped 345 previous similar messages [ 4558.947507] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4558.953837] Lustre: Skipped 345 previous similar messages [ 4558.956591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4559.634968] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4559.637957] Lustre: Skipped 345 previous similar messages [ 4559.645480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4559.653303] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4559.655962] Lustre: Skipped 345 previous similar messages [ 4560.298484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4561.018851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4561.682186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4562.261830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4562.850335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4563.398331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4564.008257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4564.662458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4565.280449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4565.949034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4566.570174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4567.287178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4567.935591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4568.461333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4569.161423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4569.889461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4570.484632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4571.012577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4571.556795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4572.143880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4572.680873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4573.399086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4574.074811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4574.738446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4575.383462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4576.112484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4576.890386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4577.756673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4578.417185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4579.193830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4579.958517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4580.748719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4581.651032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4582.422217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4583.141241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4583.925713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4584.800515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4585.672148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4586.560263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4587.666771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4588.613299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4589.522246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4590.442358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4591.629630] LustreError: 162793:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4591.641171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4591.641847] LustreError: 162793:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4592.864813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4593.764938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4594.658961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4595.606275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4596.507779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4597.390046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4598.349891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4599.158275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4599.906464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4600.674249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4601.339271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4602.050349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4602.652232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4603.406701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4604.072398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4604.802831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4605.568540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4606.446845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4607.259280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4608.073367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4608.944031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4609.847023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4610.686238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4611.649813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4612.520732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4613.411901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4614.221425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4615.048139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4616.063430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4616.826296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4617.461976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4618.174254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4619.043738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4619.781679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4620.757367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4621.715267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4622.751278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4623.760841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4624.906425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4625.889380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4626.847284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4627.785873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4628.694487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4629.470766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4630.304240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4631.082222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4631.944870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4632.748191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4633.531358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4634.287209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4635.183641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4636.101892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4636.997336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4637.777906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4638.548338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4639.283653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4640.019047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4641.004962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4641.840935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4642.631474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4643.478609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4644.425334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4645.331445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4646.403832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4647.308378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4648.188476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4649.064351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4649.897676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4650.567475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4651.269756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4652.012529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4652.731421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4653.469757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4654.290404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4654.937503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4655.643875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4656.356547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4657.026531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4657.660187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4658.389511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4659.162701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4659.826585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4660.595594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4661.396715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4662.107331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4662.796058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4663.522581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4664.267814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4665.050581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4665.883861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4666.580699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4667.516670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4668.339791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4669.148315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4669.943782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4670.689451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4671.543749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4672.309723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4673.045340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4673.748590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4674.506444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4675.145768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4675.870341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4676.545407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4677.258454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4677.878661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4678.517858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4679.132184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4679.692545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4680.335092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4681.003530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4681.770319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4682.576039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4683.354418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4684.100993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4684.888593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4685.816510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4686.863562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4687.729723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4688.541324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4689.367062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4690.197579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4691.136925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4692.216562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4693.128573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4693.905654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4694.756146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4695.508918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4696.273290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4697.152379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4697.794084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4698.591120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4699.444298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4700.492100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4701.200214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4702.052787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4702.830143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4703.692437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4704.467931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4705.362786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4706.124583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4706.903143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4707.781083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4708.563629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4709.363820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4710.069925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4710.818391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4711.699702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4712.491525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4713.346878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4714.117962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4715.092526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4715.904762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4716.777467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4717.469079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4718.323641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4719.068303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4719.820928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4720.740938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4721.582908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4722.458719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4723.170978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4724.058655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4724.881134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4725.602034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4726.348042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4727.206291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4727.963595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4728.869681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4729.610752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4730.539212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4731.450921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4732.342863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4733.187800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4734.100831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4734.854712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4735.635561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4736.534214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4737.349447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4737.993159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4738.797274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4739.511296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4740.276071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4740.953856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4741.844027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4742.696047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4743.629237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4744.435919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4745.170971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4745.976667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4746.870343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4747.760804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4748.761122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4749.679589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4750.749872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4751.591725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4752.420673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4753.221845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4754.054654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4754.834513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4755.671875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4756.431976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4757.378529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4758.173927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4758.950570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4759.711188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4760.532175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4761.466384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4762.334734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.129596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.927525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4764.700355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4765.655027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4766.419899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4767.358121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4768.110931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4768.858171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4769.650329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4770.423555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4771.227192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4772.021318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4772.771274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4773.649054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4774.390138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4774.990753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4775.736519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4776.521459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4777.310927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4778.020261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4778.749686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.529636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4780.321623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4781.177911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4781.999872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4782.812703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4783.598735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.505230] LustreError: 162793:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4784.513191] LustreError: 162793:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4784.516766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4785.206610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4785.929624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4786.609456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4787.386258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4788.109820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4788.730095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4789.481630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4790.237850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4791.134403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4791.981727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4792.873502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4793.506822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.218352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.867802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4795.550424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.230316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.946719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4797.616451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.347304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.074029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.886171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4800.617391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4801.411508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.114153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.714398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.554564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.247747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.157721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.942060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4806.697669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.518814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.282617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4809.149824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4809.953901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.792643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.710900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.639247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.457472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.378044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.434785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.416684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.436208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.409933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.323329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.194327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.056656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.925828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.787365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4823.725113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.538569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.312616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.126551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.837396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4827.657712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.450320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.300168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.138873] LustreError: 162793:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4830.142360] LustreError: 162793:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4830.152902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.984132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.894703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.676892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.738135] LustreError: 164946:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 5223.743495] LustreError: 162793:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 5621.216922] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5621.221477] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5621.226596] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5621.242715] Lustre: Skipped 343 previous similar messages [ 5626.340768] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5626.344123] Lustre: Skipped 2 previous similar messages [ 5627.028122] LustreError: 216713:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5627.151813] Lustre: server umount lustre-OST0000 complete [ 5630.434638] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5630.443119] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5635.801269] LustreError: 216914:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 5635.803218] LustreError: 216914:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5635.938619] Lustre: server umount lustre-MDT0000 complete [ 5636.584498] LustreError: 162774:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5636.601510] LustreError: 162774:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5638.342511] LustreError: 162755:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763515666 with bad export cookie 581189066428379712 [ 5638.343717] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5638.347088] LustreError: 162755:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5638.458716] LustreError: 217115:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5638.465694] LustreError: 217115:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5638.606273] Lustre: server umount lustre-MDT0001 complete [ 5644.436964] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 5645.849369] Key type lgssc unregistered [ 5646.021618] LNet: 217647:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5646.029453] LNetError: 217647:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5646.038894] LNet: Removed LNI 192.168.204.144@tcp [ 5646.462927] Key type .llcrypt unregistered [ 5646.465316] Key type ._llcrypt unregistered [ 5660.190334] Key type ._llcrypt registered [ 5660.192750] Key type .llcrypt registered [ 5660.255921] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 5660.770257] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5660.782479] alg: No test for adler32 (adler32-zlib) [ 5661.698411] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 5661.812575] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 5663.415192] Key type lgssc registered [ 5664.044228] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5669.345359] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5669.361957] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5670.649386] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5673.458684] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 5678.531589] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5678.716873] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5680.688543] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 5683.419393] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5685.621118] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5689.586869] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5689.808728] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5693.269705] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 5695.978180] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 5696.633377] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5709.507405] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5710.615691] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5710.622106] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5710.626218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.632380] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5710.632458] LustreError: 219043:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 5713.716301] Lustre: lustre-OST0000: Not available for connect from 192.168.204.44@tcp (stopping) [ 5714.404823] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5714.408250] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5714.412057] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5714.417903] Lustre: Skipped 1 previous similar message [ 5718.809221] Lustre: lustre-OST0000: Not available for connect from 192.168.204.44@tcp (stopping) [ 5718.813063] Lustre: Skipped 1 previous similar message [ 5723.928693] Lustre: lustre-OST0000: Not available for connect from 192.168.204.44@tcp (stopping) [ 5723.932255] Lustre: Skipped 2 previous similar messages [ 5726.175123] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5726.210973] LustreError: 221440:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5726.298812] Lustre: server umount lustre-OST0000 complete [ 5730.642277] LustreError: 219043:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 5756.355530] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5756.539302] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5759.578317] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 5762.405993] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5780.077603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.089872] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.090252] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5781.328254] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 17 sec [ 5783.456637] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9e17832c7000.ost_server_uuid 50 [ 5784.278882] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9e17832c7000.ost_server_uuid in FULL state after 0 sec [ 5790.690578] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5790.693475] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5790.696528] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5790.701182] Lustre: Skipped 1 previous similar message [ 5790.703981] Lustre: Skipped 2 previous similar messages [ 5791.991399] LustreError: 222786:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5792.001503] LustreError: 222786:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5792.066690] Lustre: server umount lustre-OST0000 complete [ 5795.809448] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5800.219651] LustreError: 222988:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 5800.225681] LustreError: 222988:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5800.288473] Lustre: server umount lustre-MDT0000 complete [ 5800.931601] LustreError: 219024:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5800.931601] LustreError: 221189:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5800.931612] LustreError: 221189:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5800.955027] LustreError: 219024:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5802.481681] LustreError: 219004:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763515830 with bad export cookie 13363834005574850138 [ 5802.483721] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5802.486396] LustreError: 219004:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5802.577722] LustreError: 223189:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5802.580713] LustreError: 223189:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5802.742564] Lustre: server umount lustre-MDT0001 complete [ 5808.307490] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 5809.742974] Key type lgssc unregistered [ 5809.899534] LNet: 223720:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5809.903664] LNetError: 223720:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5809.921819] LNet: Removed LNI 192.168.204.144@tcp [ 5810.291985] Key type .llcrypt unregistered [ 5810.293787] Key type ._llcrypt unregistered [ 5827.950510] Key type ._llcrypt registered [ 5827.952122] Key type .llcrypt registered [ 5828.010303] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 5828.499975] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5828.520204] alg: No test for adler32 (adler32-zlib) [ 5829.442517] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 5829.604168] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 5831.215284] Key type lgssc registered [ 5831.826738] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5836.928438] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5836.941379] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5838.221533] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5840.206942] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 5844.657722] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5844.847114] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5846.778443] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 5848.360039] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5851.978472] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5852.193520] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5855.183092] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 5859.946414] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5860.001508] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5860.118403] Lustre: lustre-OST0001: new disk, initializing [ 5860.130185] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5860.190856] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5861.215452] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5861.221480] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5861.281399] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5863.201843] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 5863.359207] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 5869.992479] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5874.544682] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5888.994241] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5888.996389] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5888.999576] Lustre: Skipped 3 previous similar messages [ 5889.002365] Lustre: Skipped 2 previous similar messages [ 5893.904193] LustreError: 228765:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 5893.959187] Lustre: server umount lustre-MDT0000 complete [ 5894.113223] LustreError: 225408:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5894.121453] LustreError: 225408:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 5896.050553] LustreError: 225389:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763515923 with bad export cookie 16257989822049833390 [ 5896.055357] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5896.080388] LustreError: 225389:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5896.087404] LustreError: 226129:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 5896.190723] LustreError: 228966:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 5896.194395] LustreError: 228966:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5896.298105] Lustre: server umount lustre-MDT0001 complete [ 5908.478797] LustreError: 229167:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5908.481409] LustreError: 229167:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5908.510268] Lustre: server umount lustre-OST0000 complete [ 5920.635290] LustreError: 229368:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5920.637587] LustreError: 229368:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5920.701352] Lustre: server umount lustre-OST0001 complete [ 5922.588264] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5922.637518] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5923.320524] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5923.390816] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5924.125032] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5924.217187] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5924.901619] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5924.968827] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5932.904607] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 5938.395309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5938.634440] LustreError: 230776:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5938.685084] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5940.443828] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 5943.780200] LustreError: 230776:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5945.376347] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5948.152542] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 5949.772836] Lustre: 231888:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5953.213545] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5953.392554] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5953.395709] Lustre: Skipped 1 previous similar message [ 5955.430045] LustreError: 232244:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5955.436821] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 5956.420325] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 5960.530912] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5963.175375] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 5963.243025] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49761) [ 5967.012631] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5968.649550] Lustre: 233733:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5981.664490] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5981.667721] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5981.678038] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5982.281436] LustreError: 234042:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5982.284375] LustreError: 234042:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5982.334259] Lustre: server umount mylustre-MDT0000 complete [ 5983.713000] LustreError: 231495:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5983.715808] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5983.720232] LustreError: 231495:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5983.733664] Lustre: Skipped 1 previous similar message [ 5984.333250] LustreError: 231890:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763516012 with bad export cookie 16257989822049836225 [ 5984.340457] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5984.341827] LustreError: 231890:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5986.514468] Lustre: server umount mylustre-MDT0001 complete [ 5999.095103] LustreError: 234444:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5999.098194] LustreError: 234444:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 5999.124127] Lustre: server umount mylustre-OST0000 complete [ 6011.176484] Lustre: server umount mylustre-OST0001 complete [ 6012.601470] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 6012.637588] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6013.113961] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 6013.157155] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6013.716950] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 6013.756441] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6014.227759] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 6014.270758] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6020.022383] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 6024.232725] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6024.444486] LustreError: 236054:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6024.454214] LustreError: 236054:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6024.488124] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6024.491318] Lustre: Skipped 1 previous similar message [ 6026.200635] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6029.677734] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6031.370926] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6032.655223] Lustre: 237162:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6035.697712] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6036.902822] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49793) [ 6038.166620] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6039.976302] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 6041.717054] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6044.223957] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6047.966477] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6049.420111] Lustre: 239011:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6057.443829] Lustre: tfs-MDT0000-lwp-OST0000: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6057.446437] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 6057.447670] Lustre: Skipped 3 previous similar messages [ 6057.449548] Lustre: Skipped 1 previous similar message [ 6062.562108] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 6062.564840] Lustre: Skipped 5 previous similar messages [ 6062.916343] LustreError: 239260:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6062.918318] LustreError: 239260:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 6062.967860] Lustre: server umount tfs-MDT0000 complete [ 6064.369251] LustreError: 236029:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763516092 with bad export cookie 16257989822049838178 [ 6064.372906] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6064.373715] LustreError: 236029:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6064.510133] Lustre: server umount tfs-MDT0001 complete [ 6075.917269] Lustre: server umount tfs-OST0000 complete [ 6087.787533] Lustre: server umount tfs-OST0001 complete [ 6089.105771] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 6089.151404] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6089.585061] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 6089.629046] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6090.093946] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 6090.121595] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6090.550406] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 6090.586435] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6096.400318] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 6100.348967] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6100.559040] LustreError: 241273:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6100.567838] LustreError: 241273:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6100.589184] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6100.592141] Lustre: Skipped 3 previous similar messages [ 6102.183424] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6105.651346] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6107.452587] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6108.678779] Lustre: 242381:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6111.508876] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6114.010737] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6115.753561] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 6117.532592] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6120.128696] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6125.293682] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49825) [ 6129.044406] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6130.511539] Lustre: 244231:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6138.251833] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 20:36:04 (1763516164) [ 6151.138731] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6151.140240] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6151.142924] Lustre: Skipped 3 previous similar messages [ 6151.145363] Lustre: Skipped 1 previous similar message [ 6156.490067] LustreError: 244599:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6156.492859] LustreError: 244599:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 6156.528048] Lustre: server umount lustre-MDT0000 complete [ 6158.037621] LustreError: 244233:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763516185 with bad export cookie 16257989822049839872 [ 6158.040766] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6158.041419] LustreError: 244233:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6161.376128] LustreError: 244245:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6161.376805] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6161.383042] LustreError: 244245:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6161.388635] Lustre: Skipped 1 previous similar message [ 6161.393837] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6161.396264] Lustre: Skipped 7 previous similar messages [ 6164.232411] Lustre: server umount lustre-MDT0001 complete [ 6180.319143] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6180.358696] Lustre: server umount lustre-OST0000 complete [ 6196.191110] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6196.264423] Lustre: server umount lustre-OST0001 complete [ 6198.762633] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 6202.863820] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 6207.334366] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6210.111221] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6212.683759] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6215.743082] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6218.824035] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6218.849994] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 6218.849994] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 6218.849994] [ 6218.863227] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 6218.974627] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6218.989038] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6219.029361] Lustre: lustre-MDT0000: new disk, initializing [ 6219.068212] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6219.070894] Lustre: Skipped 3 previous similar messages [ 6219.078797] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6220.633475] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6225.427082] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6225.458556] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6225.499936] Lustre: 247387:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6225.533181] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6225.537286] Lustre: Skipped 1 previous similar message [ 6225.577503] Lustre: lustre-MDT0001: new disk, initializing [ 6225.643381] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6225.649056] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6227.210389] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6232.102416] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6232.139033] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6232.248697] Lustre: lustre-OST0000: new disk, initializing [ 6232.251567] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6233.925656] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6233.929734] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6233.969319] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6234.574638] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6239.714817] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6239.744733] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6239.814802] Lustre: lustre-OST0001: new disk, initializing [ 6239.817490] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6241.244299] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6241.248366] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6241.290885] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6242.406825] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6247.438150] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 20:37:54 (1763516274) [ 6251.710560] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 20:37:58 (1763516278) [ 6264.800444] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6264.804118] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6264.809889] Lustre: Skipped 1 previous similar message [ 6264.813118] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6264.815468] Lustre: Skipped 3 previous similar messages [ 6269.552125] LustreError: 250814:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 6269.559461] LustreError: 250814:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 6269.586845] Lustre: server umount lustre-OST0000 complete [ 6271.969626] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6271.971395] LustreError: 249984:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6271.973781] LustreError: Skipped 1 previous similar message [ 6271.983164] LustreError: 249984:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6277.557449] Lustre: server umount lustre-MDT0000 complete [ 6279.244086] LustreError: 247380:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763516307 with bad export cookie 16257989822049841419 [ 6279.249812] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6279.251088] LustreError: 247380:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6285.444686] Lustre: server umount lustre-MDT0001 complete [ 6289.950903] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 6309.343168] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6309.417466] Lustre: server umount lustre-OST0001 complete [ 6311.788772] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 6318.880802] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 6322.994948] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6325.622069] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6328.002647] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6330.597030] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6333.404303] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6333.428222] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6333.542130] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6333.560821] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6333.604675] Lustre: lustre-MDT0000: new disk, initializing [ 6333.649379] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6335.268396] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6339.912459] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6339.937552] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6339.979869] Lustre: 254274:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6340.099633] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6341.500886] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6344.722225] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6346.092687] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6348.679234] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6348.713131] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6348.851094] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6348.854014] Lustre: Skipped 5 previous similar messages [ 6350.141855] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6350.146392] Lustre: Skipped 1 previous similar message [ 6350.148629] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6350.186420] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6351.012236] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6354.427323] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6360.548201] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6360.548499] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6360.549757] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6360.549762] Lustre: Skipped 10 previous similar messages [ 6360.550513] LustreError: Skipped 1 previous similar message [ 6360.554131] Lustre: Skipped 6 previous similar messages [ 6362.968626] Lustre: server umount lustre-OST0000 complete [ 6365.664763] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6370.521732] Lustre: server umount lustre-MDT0000 complete [ 6371.908097] LustreError: 254266:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763516399 with bad export cookie 16257989822049842896 [ 6371.910976] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6371.912052] LustreError: 254266:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6372.049126] Lustre: server umount lustre-MDT0001 complete [ 6375.983142] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 6377.148124] Key type lgssc unregistered [ 6377.286383] LNet: 257648:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6377.289544] LNetError: 257648:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6377.298340] LNet: Removed LNI 192.168.204.144@tcp [ 6377.639178] Key type .llcrypt unregistered [ 6377.640560] Key type ._llcrypt unregistered [ 6389.464947] Key type ._llcrypt registered [ 6389.466610] Key type .llcrypt registered [ 6389.519358] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 6390.002303] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6390.013977] alg: No test for adler32 (adler32-zlib) [ 6390.898195] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 6390.999665] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 6392.599181] Key type lgssc registered [ 6393.099465] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6396.164216] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6396.172088] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6397.347288] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6398.794570] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6402.177417] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6402.356577] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6404.019117] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6406.246597] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6407.654024] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6410.336501] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6410.471802] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6412.648741] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6414.703666] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6415.083057] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 6417.888867] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6417.891641] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6417.898866] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6421.932203] LustreError: 261374:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6421.975254] Lustre: server umount lustre-OST0000 complete [ 6428.128875] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6428.128935] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6428.130138] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6428.134123] Lustre: Skipped 1 previous similar message [ 6429.574794] LustreError: 261576:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 6429.577038] LustreError: 261576:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6429.617417] Lustre: server umount lustre-MDT0000 complete [ 6431.167936] LustreError: 260688:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763516458 with bad export cookie 12456802251885060003 [ 6431.172378] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6431.172555] LustreError: 260688:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6431.245388] LustreError: 261777:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6431.247666] LustreError: 261777:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6431.310960] Lustre: server umount lustre-MDT0001 complete [ 6435.481830] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 6436.623897] Key type lgssc unregistered [ 6436.769074] LNet: 262308:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6436.772049] LNetError: 262308:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6436.780344] LNet: Removed LNI 192.168.204.144@tcp [ 6437.093673] Key type .llcrypt unregistered [ 6437.094925] Key type ._llcrypt unregistered [ 6444.518969] Key type ._llcrypt registered [ 6444.520467] Key type .llcrypt registered [ 6444.571309] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 6444.994203] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6445.041942] alg: No test for adler32 (adler32-zlib) [ 6445.918742] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 6446.024654] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 6447.615199] Key type lgssc registered [ 6448.062419] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6451.825530] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6451.834361] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6467.295340] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 6472.415206] LustreError: 263459:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.144@tcp: failed processing log, type 4: rc = -110 [ 6499.039160] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6501.348916] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6503.427819] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6504.211276] LustreError: 264122:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 6504.251784] Lustre: server umount lustre-OST0000 complete [ 6510.440146] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 6511.676777] Key type lgssc unregistered [ 6511.820869] LNet: 264957:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6511.824441] LNetError: 264957:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6511.833469] LNet: Removed LNI 192.168.204.144@tcp [ 6512.174877] Key type .llcrypt unregistered [ 6512.176140] Key type ._llcrypt unregistered [ 6527.572337] Key type ._llcrypt registered [ 6527.573427] Key type .llcrypt registered [ 6527.624604] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 6528.176086] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6528.186741] alg: No test for adler32 (adler32-zlib) [ 6529.072198] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 6529.183429] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 6530.783266] Key type lgssc registered [ 6531.283916] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6537.173945] loop0: detected capacity change from 0 to 204800000 [ 6537.242239] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6537.247458] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6537.403767] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6537.426996] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6537.860692] loop0: detected capacity change from 0 to 204800000 [ 6537.930038] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6537.935496] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6538.094795] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6538.118954] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6538.577389] loop0: detected capacity change from 0 to 204800000 [ 6538.648408] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6538.653544] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6538.704635] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6538.754519] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6539.186894] loop0: detected capacity change from 0 to 204800000 [ 6539.256616] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6539.352924] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6539.743377] loop0: detected capacity change from 0 to 204800000 [ 6539.756690] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6541.401319] loop0: detected capacity change from 0 to 204800000 [ 6541.416822] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6542.913244] loop0: detected capacity change from 0 to 204800000 [ 6542.923745] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6544.365566] loop0: detected capacity change from 0 to 204800000 [ 6544.379500] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6545.879040] loop0: detected capacity change from 0 to 204800000 [ 6545.917488] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6545.938958] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6545.946205] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6545.970861] Lustre: 268333:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6545.974784] Lustre: 268333:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 6545.979098] Lustre: 268333:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6545.983256] Lustre: 268333:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 6545.987067] Lustre: 268333:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6545.991179] Lustre: 268333:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 6545.996167] Lustre: 268333:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 6546.001212] Lustre: 268333:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6546.006818] Lustre: 268333:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 6546.012676] Lustre: 268333:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 6546.017809] Lustre: 268333:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 6546.022245] Lustre: 268333:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 6546.027206] Lustre: 268333:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 6546.033062] Lustre: 268333:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 6548.859719] LustreError: 268599:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6548.915400] Lustre: server umount MGS complete [ 6549.264710] loop0: detected capacity change from 0 to 204800000 [ 6549.306691] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6549.480070] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6549.501272] Lustre: lustre-MDT0000: Aborting client recovery [ 6549.855218] loop1: detected capacity change from 0 to 204800000 [ 6549.897670] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6549.924338] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6549.954899] Lustre: 268768:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6549.960266] Lustre: 268768:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 6549.966271] Lustre: 268768:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6549.972230] Lustre: 268768:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6549.977820] Lustre: 268768:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 6549.982584] Lustre: 268768:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 6549.987963] Lustre: 268768:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6549.997558] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6550.086199] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6550.112124] Lustre: lustre-MDT0001: Aborting client recovery [ 6550.538437] loop2: detected capacity change from 0 to 204800000 [ 6550.621721] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6550.660159] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6550.683514] Lustre: 268855:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6550.688786] Lustre: 268855:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6550.695087] Lustre: 268855:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6550.701664] Lustre: 268855:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6550.709476] Lustre: 268855:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6550.713793] Lustre: 268855:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6550.719995] Lustre: 268855:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6550.780810] Lustre: Found index 0 for lustre-OST0000, updating log [ 6550.831616] Lustre: lustre-OST0000: Aborting recovery [ 6551.241548] loop3: detected capacity change from 0 to 204800000 [ 6551.306972] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6551.337837] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6551.365461] Lustre: 268942:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6551.371371] Lustre: 268942:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6551.377195] Lustre: 268942:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6551.382434] Lustre: 268942:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6551.386859] Lustre: 268942:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6551.393233] Lustre: 268942:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6551.403767] Lustre: 268942:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6551.465472] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6551.468277] Lustre: Skipped 1 previous similar message [ 6551.478172] Lustre: lustre-OST0001: Aborting recovery [ 6556.653171] LustreError: 268711:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 6559.716331] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6559.716978] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6559.724503] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 6559.729887] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 6559.734638] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 6559.737163] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 6562.053785] LustreError: 268679:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 236us: rc = -2 [ 6575.072533] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6575.073104] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6575.078033] Lustre: Skipped 2 previous similar messages [ 6575.080570] Lustre: Skipped 1 previous similar message [ 6579.523665] LustreError: 269301:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6579.526371] LustreError: 269301:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 6579.580749] Lustre: server umount lustre-MDT0000 complete [ 6579.942067] LustreError: 268660:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763516607 with bad export cookie 1848662101075383082 [ 6579.943743] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6579.947257] LustreError: 268660:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6580.191741] LustreError: 268676:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6580.191790] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6580.192962] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6580.196695] LustreError: 268676:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6580.200322] Lustre: Skipped 2 previous similar messages [ 6584.223522] LustreError: 268676:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6584.223633] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6584.231418] LustreError: 268676:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6584.234551] Lustre: Skipped 2 previous similar messages [ 6586.110641] LustreError: 269352:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 6586.112725] LustreError: 269352:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6586.181065] Lustre: server umount lustre-MDT0001 complete [ 6592.621298] LustreError: 269403:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 6592.624040] LustreError: 269403:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6592.652476] Lustre: server umount lustre-OST0000 complete [ 6607.327206] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6607.353438] LustreError: 269455:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6607.356677] LustreError: 269455:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6607.418830] Lustre: server umount lustre-OST0001 complete [ 6610.555079] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 20:43:57 (1763516637) [ 6621.270404] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 6624.913560] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 6629.088759] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6631.736251] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6634.349875] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6637.041732] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6639.929918] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6639.957649] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6640.060690] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6640.074920] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6640.118496] Lustre: lustre-MDT0000: new disk, initializing [ 6640.159323] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6640.168190] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6641.748954] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6646.219949] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6646.243156] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6646.272509] Lustre: 273000:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6646.295792] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6646.297873] Lustre: Skipped 1 previous similar message [ 6646.333279] Lustre: lustre-MDT0001: new disk, initializing [ 6646.367267] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6646.380433] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6646.384464] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6647.870400] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6651.223870] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6652.702699] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6655.245870] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6655.273705] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6655.378839] Lustre: lustre-OST0000: new disk, initializing [ 6655.380965] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6655.416542] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6657.604877] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6660.604222] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6660.609075] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6660.639477] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6660.911722] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6663.528398] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 6667.596966] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 6674.915123] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 6683.585667] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 6727.137321] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6727.138563] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6727.142679] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6727.146392] Lustre: Skipped 1 previous similar message [ 6732.257631] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6732.261539] Lustre: Skipped 1 previous similar message [ 6737.375210] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6737.398110] LustreError: 276570:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 6737.400756] LustreError: 276570:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6737.451355] Lustre: server umount lustre-OST0000 complete [ 6742.496680] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6742.497351] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6742.500607] Lustre: Skipped 2 previous similar messages [ 6742.504534] Lustre: Skipped 3 previous similar messages [ 6753.247144] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6753.312907] Lustre: server umount lustre-MDT0000 complete [ 6754.641304] LustreError: 272990:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763516782 with bad export cookie 1848662101075385791 [ 6754.644600] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6754.646624] LustreError: 272990:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6754.721031] LustreError: 276975:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6754.723638] LustreError: 276975:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 6754.781853] Lustre: server umount lustre-MDT0001 complete [ 6757.972318] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6759.523521] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6761.475778] Lustre: server umount MGS complete [ 6766.070763] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6766.226901] LustreError: 278540:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6766.234860] LustreError: 278552:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 6766.259810] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6767.663325] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6770.795867] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6772.504156] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6774.497957] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6775.923557] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6778.564537] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6780.766345] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6782.695473] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6813.666785] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6813.668281] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6813.669937] LustreError: Skipped 1 previous similar message [ 6813.675883] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6813.679242] Lustre: Skipped 5 previous similar messages [ 6819.374084] LustreError: 281218:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6819.376660] LustreError: 281218:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 6819.417491] Lustre: server umount lustre-OST0000 complete [ 6822.368774] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6822.371646] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6822.376914] Lustre: Skipped 1 previous similar message [ 6835.167362] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6835.251388] Lustre: server umount lustre-MDT0000 complete [ 6836.725350] LustreError: 280159:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763516864 with bad export cookie 1848662101075387730 [ 6836.728042] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6836.730048] LustreError: 280159:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6836.851806] Lustre: server umount lustre-MDT0001 complete [ 6840.982317] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 6842.089574] Key type lgssc unregistered [ 6842.222386] LNet: 282154:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6842.225541] LNetError: 282154:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6842.234376] LNet: Removed LNI 192.168.204.144@tcp [ 6842.560382] Key type .llcrypt unregistered [ 6842.561636] Key type ._llcrypt unregistered [ 6855.210913] Key type ._llcrypt registered [ 6855.211765] Key type .llcrypt registered [ 6855.248157] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 6861.909724] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 6862.347846] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6862.382186] alg: No test for adler32 (adler32-zlib) [ 6863.249367] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 6863.330026] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 6864.919239] Key type lgssc registered [ 6865.377531] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6868.420063] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6870.773500] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6872.872000] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6875.081688] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6877.528942] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6877.554984] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6877.560945] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6878.648703] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6878.660613] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6878.699272] Lustre: lustre-MDT0000: new disk, initializing [ 6878.725940] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6878.734407] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6880.015647] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6884.125690] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6884.150169] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6884.182162] Lustre: 286057:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6884.205543] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6884.208110] Lustre: Skipped 1 previous similar message [ 6884.245133] Lustre: lustre-MDT0001: new disk, initializing [ 6884.278683] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6884.291789] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6884.296237] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6885.644042] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6888.730658] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6889.928391] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6892.122447] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6892.146721] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6892.239718] Lustre: lustre-OST0000: new disk, initializing [ 6892.241690] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6892.273643] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6894.121422] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6894.262571] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6894.265144] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6894.299356] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6897.313575] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6900.676137] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 6904.409993] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 6910.239484] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 6921.044862] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 6966.242551] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6966.243082] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6966.249873] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6966.251583] Lustre: Skipped 1 previous similar message [ 6969.389068] LustreError: 289619:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 6969.427971] Lustre: server umount lustre-OST0000 complete [ 6971.360774] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6971.360818] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6971.361353] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6971.364434] Lustre: Skipped 2 previous similar messages [ 6971.366148] LustreError: Skipped 1 previous similar message [ 6976.480381] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6976.483356] Lustre: Skipped 2 previous similar messages [ 6976.772521] LustreError: 289820:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 6976.775308] LustreError: 289820:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6976.808806] Lustre: server umount lustre-MDT0000 complete [ 6978.067313] LustreError: 288845:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763517005 with bad export cookie 14919135483528934257 [ 6978.070775] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6978.071532] LustreError: 288845:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6978.130463] LustreError: 290021:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6978.132282] LustreError: 290021:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6978.183343] Lustre: server umount lustre-MDT0001 complete [ 6980.950796] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6982.328723] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6983.928414] LustreError: 290962:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6983.930355] LustreError: 290962:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6983.951107] Lustre: server umount MGS complete [ 6987.819379] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6988.001604] LustreError: 291598:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 6988.035323] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6989.499826] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6992.391667] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6992.510711] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6993.815381] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 6995.732503] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6997.027368] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6999.429700] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7001.600433] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7003.476111] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7038.433185] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7038.433855] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7038.435732] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7038.436111] LustreError: Skipped 1 previous similar message [ 7038.444473] Lustre: Skipped 1 previous similar message [ 7040.876546] LustreError: 294264:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7040.879014] LustreError: 294264:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 7040.916509] Lustre: server umount lustre-OST0000 complete [ 7043.552688] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7043.553150] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7043.556435] Lustre: Skipped 1 previous similar message [ 7043.558472] Lustre: Skipped 1 previous similar message [ 7048.275440] Lustre: server umount lustre-MDT0000 complete [ 7048.672039] LustreError: 291586:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7048.678082] LustreError: 291586:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 7049.571080] LustreError: 291565:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763517077 with bad export cookie 14919135483528936196 [ 7049.573680] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7049.576048] LustreError: 291565:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7049.648591] LustreError: 294667:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7049.650470] LustreError: 294667:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 7049.705261] Lustre: server umount lustre-MDT0001 complete [ 7053.318768] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 7054.403628] Key type lgssc unregistered [ 7054.543474] LNet: 295198:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7054.546144] LNetError: 295198:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7054.554323] LNet: Removed LNI 192.168.204.144@tcp [ 7054.857486] Key type .llcrypt unregistered [ 7054.858738] Key type ._llcrypt unregistered [ 7061.814990] Key type ._llcrypt registered [ 7061.815947] Key type .llcrypt registered [ 7061.849345] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 7067.426313] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 7067.790545] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7067.811162] alg: No test for adler32 (adler32-zlib) [ 7068.660440] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 7068.740573] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 7070.319161] Key type lgssc registered [ 7070.675257] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7073.401350] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7075.358330] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7078.252490] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 7081.143855] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7083.400867] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7083.416748] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7083.420771] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7084.503909] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7084.517476] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7084.559536] Lustre: lustre-MDT0000: new disk, initializing [ 7084.584254] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7084.590446] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7085.828861] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7089.896348] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7089.921841] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7090.009495] Lustre: lustre-OST0000: new disk, initializing [ 7090.011363] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7090.012793] Lustre: Skipped 1 previous similar message [ 7090.032923] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7090.878929] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 7090.881734] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 7090.889460] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 7091.843623] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7275.456953] LustreError: 300111:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 7275.488436] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7275.587353] Lustre: server umount lustre-MDT0000 complete [ 7276.900416] LustreError: 299160:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763517304 with bad export cookie 11305839461041596885 [ 7276.903532] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7276.903973] LustreError: 299160:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7282.987334] LustreError: 300313:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 7282.990112] LustreError: 300313:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7283.037244] Lustre: server umount lustre-OST0000 complete [ 7291.067076] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 7294.139177] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 7297.587294] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7299.729309] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7301.765138] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7303.850176] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7306.213963] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7306.238803] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7306.323817] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7306.335895] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7306.373255] Lustre: lustre-MDT0000: new disk, initializing [ 7306.398524] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7306.405716] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7307.724477] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7311.722712] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7311.745151] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7311.779846] Lustre: 303350:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7311.810189] LustreError: 303362:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7311.813941] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7311.816059] LustreError: 303362:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7311.818539] Lustre: Skipped 1 previous similar message [ 7311.856293] Lustre: lustre-MDT0001: new disk, initializing [ 7311.887285] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7311.901461] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7311.904912] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7313.202271] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7316.130889] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7317.382160] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7319.795122] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7319.814802] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7319.916736] Lustre: lustre-OST0000: new disk, initializing [ 7319.919701] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7319.952062] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7321.947276] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7325.091250] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7326.574582] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7327.226420] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7327.230032] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7327.266067] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7327.691809] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 7329.081478] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 7329.150661] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 7332.322788] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7332.325788] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7332.329070] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7332.331331] Lustre: Skipped 1 previous similar message [ 7335.854116] LustreError: 306297:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 7335.856771] LustreError: 306297:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7335.900047] Lustre: server umount lustre-OST0000 complete [ 7337.440316] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7337.440427] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7337.440873] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7337.444070] Lustre: Skipped 2 previous similar messages [ 7337.446200] LustreError: Skipped 1 previous similar message [ 7342.560634] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7342.563789] Lustre: Skipped 2 previous similar messages [ 7343.296766] LustreError: 306498:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 7343.300405] LustreError: 306498:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7343.331795] Lustre: server umount lustre-MDT0000 complete [ 7344.628464] LustreError: 303342:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763517372 with bad export cookie 11305839461044537662 [ 7344.632211] LustreError: 303342:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7344.633124] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7344.763082] Lustre: server umount lustre-MDT0001 complete [ 7345.929626] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 7346.520281] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 20:56:13 (1763517373) [ 7348.675810] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7348.813297] LustreError: 307272:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7348.835589] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7350.099435] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7352.933833] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7354.259667] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7355.885152] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7356.982673] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7359.250460] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7361.187093] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7362.832582] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7365.015302] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7365.038243] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7365.091062] Lustre: lustre-OST0001: new disk, initializing [ 7365.094274] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7365.125317] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7365.128246] Lustre: Skipped 2 previous similar messages [ 7365.761404] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7365.764359] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7365.779620] LustreError: 309889:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 7366.818811] LustreError: 309889:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 7366.821517] LustreError: 309889:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 7367.049465] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7367.846238] LustreError: 309889:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 7367.849313] LustreError: 309889:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 7369.893148] LustreError: 309897:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 7369.898116] LustreError: 309897:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 7370.771980] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7373.992338] LustreError: 309897:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 7373.997814] LustreError: 309897:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 7374.924631] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 4 sec [ 7376.183200] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9e17832c6800.ost_server_uuid 50 [ 7376.707559] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9e17832c6800.ost_server_uuid in FULL state after 0 sec [ 7377.876912] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9e17832c6800.ost_server_uuid 50 [ 7378.396373] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9e17832c6800.ost_server_uuid in FULL state after 0 sec [ 7379.113456] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7394.787461] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7394.788030] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7394.789474] LustreError: Skipped 1 previous similar message [ 7394.793320] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7394.795735] Lustre: Skipped 1 previous similar message [ 7396.206150] LustreError: 311101:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 7396.208724] LustreError: 311101:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 7396.229616] Lustre: server umount lustre-OST0001 complete [ 7398.367991] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7399.905119] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7399.906634] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7399.906740] LustreError: 308866:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7399.909217] Lustre: Skipped 2 previous similar messages [ 7399.910906] Lustre: Skipped 2 previous similar messages [ 7404.116359] Lustre: server umount lustre-OST0000 complete [ 7409.631940] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7409.635235] LustreError: Skipped 1 previous similar message [ 7409.637236] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7409.642662] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7409.644501] Lustre: Skipped 1 previous similar message [ 7411.588810] LustreError: 311503:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 7411.592225] LustreError: 311503:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 7411.626265] Lustre: server umount lustre-MDT0000 complete [ 7412.937593] LustreError: 307252:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763517440 with bad export cookie 11305839461044538544 [ 7412.940697] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7412.941735] LustreError: 307252:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7413.066698] Lustre: server umount lustre-MDT0001 complete [ 7416.837435] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 7417.867985] Key type lgssc unregistered [ 7417.989344] LNet: 312238:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7417.991716] LNetError: 312238:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7418.001308] LNet: Removed LNI 192.168.204.144@tcp [ 7418.288234] Key type .llcrypt unregistered [ 7418.289270] Key type ._llcrypt unregistered [ 7427.027956] Key type ._llcrypt registered [ 7427.029198] Key type .llcrypt registered [ 7427.060332] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 7427.454915] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7427.463950] alg: No test for adler32 (adler32-zlib) [ 7428.314228] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 7428.392455] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 7429.967162] Key type lgssc registered [ 7430.316544] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7432.728215] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7432.734253] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7433.883723] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7435.050228] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7436.768194] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7439.068172] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7439.179878] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7440.386090] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7443.463644] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7443.576790] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7445.584954] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7447.400119] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7448.676548] LustreError: 315166:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7448.686144] LustreError: 315166:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7449.775509] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7449.845043] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7451.743502] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7453.570782] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 7458.281256] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 7458.283810] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7460.617828] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 7460.698149] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 7540.196686] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7540.197078] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7540.206566] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7540.208597] Lustre: Skipped 1 previous similar message [ 7543.406099] LustreError: 317038:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 7543.425379] Lustre: server umount lustre-OST0001 complete [ 7545.312788] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7545.313449] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7545.314538] LustreError: 315166:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7545.314742] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7545.328305] Lustre: Skipped 2 previous similar messages [ 7550.432375] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7550.433389] LustreError: 315166:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7550.434515] Lustre: Skipped 1 previous similar message [ 7550.440156] LustreError: 315166:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7551.084097] LustreError: 317240:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 7551.086918] LustreError: 317240:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7551.122512] Lustre: server umount lustre-OST0000 complete [ 7555.552922] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7555.553813] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7555.557689] Lustre: Skipped 1 previous similar message [ 7558.530509] LustreError: 317441:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 7558.533423] LustreError: 317441:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7558.566281] Lustre: server umount lustre-MDT0000 complete [ 7559.870169] LustreError: 317040:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763517587 with bad export cookie 6023605390228807869 [ 7559.872755] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7559.875147] LustreError: 317040:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7560.011460] Lustre: server umount lustre-MDT0001 complete [ 7563.961694] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 7565.023718] Key type lgssc unregistered [ 7565.162828] LNet: 318174:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7565.165662] LNetError: 318174:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7565.175577] LNet: Removed LNI 192.168.204.144@tcp [ 7565.473300] Key type .llcrypt unregistered [ 7565.474795] Key type ._llcrypt unregistered [ 7584.290398] Key type ._llcrypt registered [ 7584.291348] Key type .llcrypt registered [ 7584.330597] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 7584.672254] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7584.721306] alg: No test for adler32 (adler32-zlib) [ 7585.577219] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 7585.672251] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 7587.255162] Key type lgssc registered [ 7587.632021] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7590.936825] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7590.942702] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7592.110283] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7593.507679] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7596.597408] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7596.737676] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7598.119849] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7599.157522] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7601.570384] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7601.697747] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7602.723387] LustreError: 322065:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7602.726655] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 7602.730316] LustreError: 322065:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7603.509562] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7606.214713] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7606.294586] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7607.338176] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 7608.095787] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7612.126863] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7617.843241] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7627.746886] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7627.748620] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7627.749014] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7627.750525] Lustre: Skipped 3 previous similar messages [ 7632.828852] LustreError: 323831:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 7632.858382] Lustre: server umount lustre-MDT0000 complete [ 7632.867853] LustreError: 320421:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7632.874816] LustreError: 320421:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7634.072853] LustreError: 320402:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763517661 with bad export cookie 9975585620858433948 [ 7634.076663] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7634.077849] LustreError: 320402:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7634.148754] LustreError: 324044:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 7634.151116] LustreError: 324044:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 7634.200385] Lustre: server umount lustre-MDT0001 complete [ 7646.195149] LustreError: 324244:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 7646.197715] LustreError: 324244:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 7646.214142] Lustre: server umount lustre-OST0000 complete [ 7657.716168] LustreError: 324447:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 7657.718202] LustreError: 324447:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7657.759071] Lustre: server umount lustre-OST0001 complete [ 7663.383657] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 7666.989318] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7667.140837] LustreError: 325623:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7667.163196] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7668.510712] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7671.490735] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7672.908602] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7673.817224] Lustre: 326731:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7675.921543] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7676.051360] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7676.053414] Lustre: Skipped 1 previous similar message [ 7677.090728] LustreError: 327089:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7677.095073] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 7678.104318] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7681.093614] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7682.219266] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 7683.180189] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7686.493946] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7687.771438] Lustre: 328579:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7697.376194] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7697.378902] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7697.382985] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7697.384543] Lustre: Skipped 3 previous similar messages [ 7699.848195] LustreError: 328685:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 7699.850586] LustreError: 328685:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7699.877759] Lustre: server umount lustre-MDT0000 complete [ 7701.157386] LustreError: 326734:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763517728 with bad export cookie 9975585620858435635 [ 7701.161790] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7701.162602] LustreError: 326734:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7701.306820] Lustre: server umount lustre-MDT0001 complete [ 7713.204179] LustreError: 329088:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 7713.206449] LustreError: 329088:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 7713.225702] Lustre: server umount lustre-OST0000 complete [ 7724.769226] Lustre: server umount lustre-OST0001 complete [ 7725.977572] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 7726.530085] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 7727.117528] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 21:02:33 (1763517753) [ 7734.631649] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 7738.083587] loop0: detected capacity change from 0 to 1099511627776 [ 7738.230303] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 7738.232604] *** VALIDATE xfs *** [ 7738.237837] XFS (loop0): Mounting V5 Filesystem [ 7738.249530] XFS (loop0): Ending clean mount [ 7738.675465] loop1: detected capacity change from 0 to 18691697672192 [ 7754.203076] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7760.109194] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 7763.245329] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 7766.599815] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7768.780401] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7770.958400] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7773.185248] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7775.503733] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7775.525511] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7775.605873] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7775.616801] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7775.652266] Lustre: lustre-MDT0000: new disk, initializing [ 7775.673772] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7775.675357] Lustre: Skipped 1 previous similar message [ 7775.681187] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7776.962923] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7781.034752] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7781.056140] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7781.085872] Lustre: 333656:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7781.108291] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7781.111192] Lustre: Skipped 1 previous similar message [ 7781.146656] Lustre: lustre-MDT0001: new disk, initializing [ 7781.189763] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7781.194626] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7782.463213] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7785.669634] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7786.885777] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7788.983838] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7789.006761] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7789.095535] Lustre: lustre-OST0000: new disk, initializing [ 7789.098074] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7790.264548] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7790.267235] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7790.305661] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7790.963311] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7793.942135] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7795.362952] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7795.432372] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 7796.647393] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 7796.716781] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 7799.406149] LustreError: 336594:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 7799.408958] LustreError: 336594:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 7799.456407] Lustre: server umount lustre-OST0000 complete [ 7800.801870] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7800.802734] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7800.804900] Lustre: Skipped 4 previous similar messages [ 7800.806444] Lustre: Skipped 3 previous similar messages [ 7805.920559] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7805.923673] Lustre: Skipped 2 previous similar messages [ 7811.040473] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7811.042399] Lustre: Skipped 1 previous similar message [ 7815.135198] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7815.211975] Lustre: server umount lustre-MDT0000 complete [ 7816.160163] LustreError: 333667:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7816.165797] LustreError: 333667:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7816.548580] LustreError: 334437:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763517844 with bad export cookie 9975585620858437189 [ 7816.552122] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7816.554271] LustreError: 334437:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7826.965679] Lustre: server umount lustre-MDT0001 complete [ 7829.727750] XFS (loop0): Unmounting Filesystem [ 7831.359774] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 21:04:18 (1763517858) [ 7833.848157] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7834.017845] LustreError: 337816:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7834.050649] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7834.053845] Lustre: Skipped 2 previous similar messages [ 7835.722149] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7838.741543] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7840.087666] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7841.795639] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7842.918962] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7845.166873] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7847.235499] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7849.169864] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7863.776916] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7863.778604] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7863.781722] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7863.784703] Lustre: Skipped 1 previous similar message [ 7863.790221] Lustre: Skipped 1 previous similar message [ 7864.686065] LustreError: 340180:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7864.687897] LustreError: 340180:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 7864.722732] Lustre: server umount lustre-OST0000 complete [ 7868.116401] Lustre: server umount lustre-MDT0000 complete [ 7868.898765] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7868.903777] LustreError: 339223:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7868.908292] LustreError: 339223:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7869.476520] LustreError: 340381:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763517897 with bad export cookie 9975585620858438071 [ 7869.479946] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7869.482634] LustreError: 340381:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7869.622782] Lustre: server umount lustre-MDT0001 complete [ 7873.549645] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 7874.772293] Key type lgssc unregistered [ 7874.928502] LNet: 341113:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7874.931089] LNetError: 341113:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7874.939473] LNet: Removed LNI 192.168.204.144@tcp [ 7875.319202] Key type .llcrypt unregistered [ 7875.320621] Key type ._llcrypt unregistered [ 7886.234730] Key type ._llcrypt registered [ 7886.236531] Key type .llcrypt registered [ 7886.274293] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 7886.703113] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7886.741938] alg: No test for adler32 (adler32-zlib) [ 7887.595112] Lustre: Lustre: Build Version: 2.16.61_43_g35e72c9 [ 7887.683701] LNet: Added LNI 192.168.204.144@tcp [8/256/0/180] [ 7889.271231] Key type lgssc registered [ 7889.679938] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7892.373432] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7892.379042] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7893.539433] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7894.873459] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7897.826461] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7897.951746] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7899.356857] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7901.256544] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7902.547434] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7904.953526] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7905.085181] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7907.068349] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7908.946730] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7928.799875] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 7928.801871] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7928.805524] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7929.625315] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.44@tcp (stopping) [ 7929.826597] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7933.246301] LustreError: 344957:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7933.272817] Lustre: server umount lustre-MDT0001 complete [ 7934.288950] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7934.738941] LustreError: 344721:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.44@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7934.744108] LustreError: 344721:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7936.979614] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7937.015560] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 7937.018045] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7937.032524] Lustre: 342628:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7937.162543] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7938.619771] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 7942.634954] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 7945.190820] LustreError: 346085:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 7945.193540] LustreError: 346085:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 7945.196580] LustreError: 342630:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 7945.214479] LustreError: 346088:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.204.144@tcp: cfg command failed: rc = -17 [ 7945.219505] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 7945.219505] [ 8241.946244] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.44@tcp (stopping) [ 8241.949164] Lustre: Skipped 2 previous similar messages [ 8244.706069] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 8244.706594] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8244.715886] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8244.718326] Lustre: Skipped 1 previous similar message [ 8247.103854] LustreError: 357475:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 8247.105761] LustreError: 357475:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8247.132853] Lustre: server umount lustre-MDT0001 complete [ 8248.169433] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8249.825921] LustreError: 342638:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8249.834073] LustreError: 342638:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8250.795164] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8250.825213] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 8250.826495] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8250.836213] Lustre: 342628:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8250.953981] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8252.271259] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 8255.973119] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 8258.535927] LustreError: 358615:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 8258.539587] LustreError: 358615:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 8258.543469] LustreError: 342630:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 8258.545018] LustreError: 342630:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 8258.562981] LustreError: 358624:0:(obd_config.c:689:class_setup()) Device 8 already setup (type osp) [ 8258.565036] LustreError: 358624:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.204.144@tcp: cfg command failed: rc = -17 [ 8258.567653] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.144@tcp [ 8258.567653] [ 8558.050261] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 8558.051554] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8558.054338] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8558.056816] Lustre: Skipped 2 previous similar messages [ 8558.058593] Lustre: Skipped 1 previous similar message [ 8561.795539] LustreError: 370092:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 8561.798725] LustreError: 370092:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8561.825808] Lustre: server umount lustre-MDT0001 complete [ 8562.866466] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8563.170969] LustreError: 343362:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8563.175166] LustreError: 343362:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8565.501625] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8565.534413] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 8565.536172] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8565.548955] Lustre: 342628:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8565.671284] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 8567.107823] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 8570.856984] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 8575.984150] LustreError: 371320:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 8575.988623] LustreError: 371320:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 8575.993515] LustreError: 342630:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 8575.997050] LustreError: 342630:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 8576.017368] LustreError: 371323:0:(lod_lov.c:207:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 8576.024768] LustreError: 371323:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.204.144@tcp: cfg command failed: rc = -114 [ 8576.030233] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 8576.030233] [ 8883.169876] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8883.170964] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8883.172900] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8883.177374] Lustre: Skipped 1 previous similar message [ 8883.180103] Lustre: Skipped 3 previous similar messages [ 8887.167092] LustreError: 382777:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 8887.168900] LustreError: 382777:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8887.200466] Lustre: server umount lustre-MDT0000 complete [ 8888.288901] LustreError: 342663:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8888.293546] LustreError: 342663:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8888.530369] LustreError: 342620:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763518916 with bad export cookie 5355543485462840450 [ 8888.534794] LustreError: 342620:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8888.535796] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8893.407929] LustreError: 342635:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8893.408431] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8893.422558] Lustre: Skipped 1 previous similar message [ 8894.653937] LustreError: 382978:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 8894.655860] LustreError: 382978:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8894.710903] Lustre: server umount lustre-MDT0001 complete [ 8896.128797] Lustre: server umount lustre-OST0000 complete [ 8899.097098] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 8902.292200] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 8905.869893] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8908.096395] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8910.189922] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8912.432277] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8914.701870] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 21:22:21 (1763518941) [ 8916.966905] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8916.986691] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8917.088116] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8917.099758] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8917.143236] Lustre: lustre-MDT0000: new disk, initializing [ 8917.168540] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8917.175525] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8918.463779] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 8922.767105] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8922.796930] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8922.834748] Lustre: 385708:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8922.862055] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8922.863651] Lustre: Skipped 1 previous similar message [ 8922.905013] Lustre: lustre-MDT0001: new disk, initializing [ 8922.941482] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8922.956848] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8922.961872] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8924.432879] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 8927.794660] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8929.057739] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8931.376619] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8931.405433] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8931.511486] Lustre: lustre-OST0000: new disk, initializing [ 8931.514571] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8931.548030] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8932.476861] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8932.480277] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8932.518658] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8933.511494] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_default_debug -1 all [ 8936.763460] Lustre: DEBUG MARKER: oleg444-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8943.073401] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8943.075233] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8943.079980] Lustre: Skipped 1 previous similar message [ 8943.082993] Lustre: Skipped 1 previous similar message [ 8945.544948] LustreError: 388154:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 8945.547977] LustreError: 388154:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 8945.587049] Lustre: server umount lustre-MDT0000 complete [ 8946.925401] LustreError: 385701:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763518974 with bad export cookie 5355543485462842403 [ 8946.929642] LustreError: 386501:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 8946.931037] LustreError: 385701:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8946.931058] LustreError: MGC192.168.204.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8947.071634] Lustre: server umount lustre-MDT0001 complete [ 8958.686566] Lustre: server umount lustre-OST0000 complete [ 8968.529285] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing set_hostid [ 8971.950354] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing load_modules_local [ 8975.419804] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8977.776871] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8979.885096] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8982.167772] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8982.740171] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 8777 sec ========== 21:23:29 (1763519009) [ 8983.349890] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 21:23:30 (1763519010) === [ 8984.465346] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 21:23:31 (1763519011) === [ 8997.892258] Lustre: DEBUG MARKER: oleg444-server.virtnet: executing unload_modules_local [ 8998.871165] Key type lgssc unregistered [ 8998.988381] LNet: 393658:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8998.990770] LNetError: 393658:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8999.001325] LNet: Removed LNI 192.168.204.144@tcp [ 8999.291632] Key type .llcrypt unregistered [ 8999.293320] Key type ._llcrypt unregistered