[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 497531187 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f5b30-0x000f5b3f] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F5950 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE1BB7 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE1A53 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 001A13 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE1AC7 000090 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE1B57 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE1B8F 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe1a53-0xbffe1ac6] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe1a52] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe1ac7-0xbffe1b56] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe1b57-0xbffe1b8e] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe1b8f-0xbffe1bb6] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524576K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001013] APIC: Switch to symmetric I/O mode setup [ 0.003128] x2apic enabled [ 0.004007] Switched APIC routing to physical x2apic. [ 0.005011] kvm-guest: setup PV IPIs [ 0.007905] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008022] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009014] pid_max: default: 32768 minimum: 301 [ 0.010131] LSM: Security Framework initializing [ 0.011056] Yama: becoming mindful. [ 0.012041] SELinux: Initializing. [ 0.013066] *** VALIDATE selinux *** [ 0.021828] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026285] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027157] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029022] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030122] *** VALIDATE tmpfs *** [ 0.031461] *** VALIDATE proc *** [ 0.032264] *** VALIDATE cgroup *** [ 0.033011] *** VALIDATE cgroup2 *** [ 0.035142] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036175] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037007] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038029] Spectre V2 : User space: Vulnerable [ 0.039007] Speculative Store Bypass: Vulnerable [ 0.042336] debug: unmapping init [mem 0xffffffff9b459000-0xffffffff9b460fff] [ 0.045262] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.046727] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.047029] ... version: 2 [ 0.048012] ... bit width: 48 [ 0.049012] ... generic registers: 4 [ 0.050014] ... value mask: 0000ffffffffffff [ 0.051016] ... max period: 00007fffffffffff [ 0.052017] ... fixed-purpose events: 3 [ 0.053012] ... event mask: 000000070000000f [ 0.054291] rcu: Hierarchical SRCU implementation. [ 0.056330] smp: Bringing up secondary CPUs ... [ 0.057548] x86: Booting SMP configuration: [ 0.058026] .... node #0, CPUs: #1 #2 #3 [ 0.063073] smp: Brought up 1 node, 4 CPUs [ 0.065017] smpboot: Max logical packages: 1 [ 0.066009] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.143020] node 0 deferred pages initialised in 71ms [ 0.147009] devtmpfs: initialized [ 0.148229] x86/mm: Memory block size: 128MB [ 0.150660] gcov: version magic: 0x41383552 [ 0.152341] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.153082] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.154258] pinctrl core: initialized pinctrl subsystem [ 0.155217] [ 0.155823] ************************************************************* [ 0.156015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.157016] ** ** [ 0.158016] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.159016] ** ** [ 0.160016] ** This means that this kernel is built to expose internal ** [ 0.161014] ** IOMMU data structures, which may compromise security on ** [ 0.162015] ** your system. ** [ 0.163011] ** ** [ 0.164013] ** If you see this message and you are not debugging the ** [ 0.165013] ** kernel, report this immediately to your vendor! ** [ 0.166013] ** ** [ 0.167012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.168014] ************************************************************* [ 0.169604] NET: Registered protocol family 16 [ 0.170441] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.171064] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.172061] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.173506] cpuidle: using governor menu [ 0.175623] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.177569] PCI: Using configuration type 1 for base access [ 0.180117] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.188109] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.189016] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.191111] cryptd: max_cpu_qlen set to 1000 [ 0.192238] ACPI: Added _OSI(Module Device) [ 0.194014] ACPI: Added _OSI(Processor Device) [ 0.195014] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.197016] ACPI: Added _OSI(Processor Aggregator Device) [ 0.201432] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.208106] ACPI: Interpreter enabled [ 0.210071] ACPI: PM: (supports S0 S3 S4 S5) [ 0.212016] ACPI: Using IOAPIC for interrupt routing [ 0.214163] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.218402] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.228747] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.232045] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.234022] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.238080] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.243292] acpiphp: Slot [2] registered [ 0.245159] acpiphp: Slot [3] registered [ 0.246072] acpiphp: Slot [4] registered [ 0.248127] acpiphp: Slot [5] registered [ 0.249200] acpiphp: Slot [6] registered [ 0.250000] acpiphp: Slot [7] registered [ 0.250000] acpiphp: Slot [8] registered [ 0.253136] acpiphp: Slot [9] registered [ 0.254112] acpiphp: Slot [10] registered [ 0.256139] acpiphp: Slot [11] registered [ 0.258103] acpiphp: Slot [12] registered [ 0.259104] acpiphp: Slot [13] registered [ 0.261092] acpiphp: Slot [14] registered [ 0.262099] acpiphp: Slot [15] registered [ 0.264104] acpiphp: Slot [16] registered [ 0.265108] acpiphp: Slot [17] registered [ 0.267130] acpiphp: Slot [18] registered [ 0.269109] acpiphp: Slot [19] registered [ 0.270096] acpiphp: Slot [20] registered [ 0.272110] acpiphp: Slot [21] registered [ 0.273109] acpiphp: Slot [22] registered [ 0.275105] acpiphp: Slot [23] registered [ 0.276111] acpiphp: Slot [24] registered [ 0.278120] acpiphp: Slot [25] registered [ 0.280111] acpiphp: Slot [26] registered [ 0.281101] acpiphp: Slot [27] registered [ 0.283106] acpiphp: Slot [28] registered [ 0.284143] acpiphp: Slot [29] registered [ 0.286133] acpiphp: Slot [30] registered [ 0.288138] acpiphp: Slot [31] registered [ 0.289050] PCI host bridge to bus 0000:00 [ 0.291020] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.293030] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.295021] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.298020] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.299019] pci_bus 0000:00: root bus resource [mem 0x180000000-0x1ffffffff window] [ 0.302051] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.304219] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.308444] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.312049] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.321686] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.326619] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.329019] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.332016] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.334017] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.336446] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.338769] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.341042] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.343654] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.347904] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.359911] pci 0000:00:02.0: reg 0x20: [mem 0xfebe4000-0xfebe7fff 64bit pref] [ 0.364016] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.370252] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.379031] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.386019] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.406055] pci 0000:00:05.0: reg 0x20: [mem 0xfebe8000-0xfebebfff 64bit pref] [ 0.421638] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.430019] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.435019] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.457028] pci 0000:00:06.0: reg 0x20: [mem 0xfebec000-0xfebeffff 64bit pref] [ 0.470945] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.482030] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.495021] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.511023] pci 0000:00:07.0: reg 0x20: [mem 0xfebf0000-0xfebf3fff 64bit pref] [ 0.524000] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.530022] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.535016] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.552033] pci 0000:00:08.0: reg 0x20: [mem 0xfebf4000-0xfebf7fff 64bit pref] [ 0.561712] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.569027] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.578033] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.600028] pci 0000:00:09.0: reg 0x20: [mem 0xfebf8000-0xfebfbfff 64bit pref] [ 0.609169] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.614018] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.621026] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.636024] pci 0000:00:0a.0: reg 0x20: [mem 0xfebfc000-0xfebfffff 64bit pref] [ 0.647394] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.651523] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.654478] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.656449] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.659265] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.665249] iommu: Default domain type: Passthrough [ 0.668633] SCSI subsystem initialized [ 0.670206] ACPI: bus type USB registered [ 0.671142] usbcore: registered new interface driver usbfs [ 0.674160] usbcore: registered new interface driver hub [ 0.676136] usbcore: registered new device driver usb [ 0.678223] pps_core: LinuxPPS API ver. 1 registered [ 0.680017] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.683068] PTP clock support registered [ 0.686045] EDAC MC: Ver: 3.0.0 [ 0.687293] PCI: Using ACPI for IRQ routing [ 0.688000] NetLabel: Initializing [ 0.690018] NetLabel: domain hash size = 128 [ 0.691010] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.693099] NetLabel: unlabeled traffic allowed by default [ 0.695131] vgaarb: loaded [ 0.696398] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.698019] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.703350] clocksource: Switched to clocksource kvm-clock [ 0.808925] VFS: Disk quotas dquot_6.6.0 [ 0.810540] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.813165] *** VALIDATE ramfs *** [ 0.814224] *** VALIDATE hugetlbfs *** [ 0.816570] pnp: PnP ACPI init [ 0.818861] pnp: PnP ACPI: found 6 devices [ 0.843622] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.847473] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.849621] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.852606] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.856030] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.858993] pci_bus 0000:00: resource 8 [mem 0x180000000-0x1ffffffff window] [ 0.861923] NET: Registered protocol family 2 [ 0.867122] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.872293] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.876282] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.881942] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.885585] TCP: Hash tables configured (established 65536 bind 65536) [ 0.889047] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.892495] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.895638] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.898634] NET: Registered protocol family 1 [ 0.901331] RPC: Registered named UNIX socket transport module. [ 0.903262] RPC: Registered udp transport module. [ 0.905049] RPC: Registered tcp transport module. [ 0.906935] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.910345] NET: Registered protocol family 44 [ 0.912099] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.914486] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.916555] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.918849] PCI: CLS 0 bytes, default 64 [ 0.921228] Unpacking initramfs... [ 2.264270] debug: unmapping init [mem 0xffff9856bcc54000-0xffff9856bffbffff] [ 2.268430] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.270300] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.272699] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.787447] Initialise system trusted keyrings [ 2.789449] Key type blacklist registered [ 2.791604] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.800732] zbud: loaded [ 2.803941] *** VALIDATE nfs *** [ 2.805545] *** VALIDATE nfs4 *** [ 2.806771] pstore: using deflate compression [ 2.809820] Platform Keyring initialized [ 2.925059] NET: Registered protocol family 38 [ 2.926824] Key type asymmetric registered [ 2.928832] Asymmetric key parser 'x509' registered [ 2.931764] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.935274] io scheduler mq-deadline registered [ 2.937131] io scheduler kyber registered [ 2.939021] io scheduler bfq registered [ 2.946196] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.948424] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.950555] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.953360] ACPI: Power Button [PWRF] [ 3.046582] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.141323] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.336774] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.439790] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.628041] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.657784] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.691514] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.697248] Non-volatile memory driver v1.3 [ 3.699121] Linux agpgart interface v0.103 [ 3.739558] virtio_blk virtio1: [vda] 133640 512-byte logical blocks (68.4 MB/65.3 MiB) [ 3.741959] vda: detected capacity change from 0 to 68423680 [ 3.765231] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.768638] vdb: detected capacity change from 0 to 1073741824 [ 3.785329] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.788396] vdc: detected capacity change from 0 to 2621440000 [ 3.802512] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.805495] vdd: detected capacity change from 0 to 2621440000 [ 3.821686] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.824740] vde: detected capacity change from 0 to 4294967296 [ 3.839672] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.842912] vdf: detected capacity change from 0 to 4294967296 [ 3.849935] libphy: Fixed MDIO Bus: probed [ 3.859947] usbcore: registered new interface driver usbserial_generic [ 3.862834] usbserial: USB Serial support registered for generic [ 3.866932] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.871782] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.873805] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.876365] mousedev: PS/2 mouse device common for all mice [ 3.880097] rtc_cmos 00:05: RTC can wake from S4 [ 3.881852] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.886698] rtc_cmos 00:05: registered as rtc0 [ 3.891925] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.895502] intel_pstate: CPU model not supported [ 3.897331] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.900762] hid: raw HID events driver (C) Jiri Kosina [ 3.903394] usbcore: registered new interface driver usbhid [ 3.905580] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.905973] usbhid: USB HID core driver [ 3.910271] drop_monitor: Initializing network drop monitor service [ 3.912584] Initializing XFRM netlink socket [ 3.914738] NET: Registered protocol family 10 [ 3.918046] Segment Routing with IPv6 [ 3.919540] NET: Registered protocol family 17 [ 3.921755] mpls_gso: MPLS GSO support [ 3.926714] RAS: Correctable Errors collector initialized. [ 3.929038] AVX version of gcm_enc/dec engaged. [ 3.930586] AES CTR mode by8 optimization enabled [ 4.008428] sched_clock: Marking stable (4008405320, 0)->(4977292583, -968887263) [ 4.012689] registered taskstats version 1 [ 4.014635] Loading compiled-in X.509 certificates [ 4.015799] zswap: loaded using pool lzo/zbud [ 4.041136] Key type big_key registered [ 4.055847] Key type encrypted registered [ 4.058369] ima: No TPM chip found, activating TPM-bypass! [ 4.060603] ima: Allocated hash algorithm: sha1 [ 4.062650] ima: No architecture policies found [ 4.064135] evm: Initialising EVM extended attributes: [ 4.071270] evm: security.selinux [ 4.072628] evm: security.ima [ 4.073796] evm: security.capability [ 4.075122] evm: HMAC attrs: 0x1 [ 4.081192] rtc_cmos 00:05: setting system clock to 2025-11-14 20:48:15 UTC (1763153295) [ 4.091270] debug: unmapping init [mem 0xffffffff9c403000-0xffffffff9c5fffff] [ 4.095183] debug: unmapping init [mem 0xffffffff9b182000-0xffffffff9b458fff] [ 4.100467] Write protecting the kernel read-only data: 28672k [ 4.103667] debug: unmapping init [mem 0xffffffff99803000-0xffffffff999fffff] [ 4.106670] debug: unmapping init [mem 0xffffffff9a114000-0xffffffff9a1fffff] [ 4.145425] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.153833] systemd[1]: Detected virtualization kvm. [ 4.155533] systemd[1]: Detected architecture x86-64. [ 4.157576] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.182410] systemd[1]: No hostname configured. [ 4.185610] systemd[1]: Set hostname to . [ 4.189346] random: systemd: uninitialized urandom read (16 bytes read) [ 4.193880] systemd[1]: Initializing machine ID from random generator. [ 4.226807] random: ln: uninitialized urandom read (6 bytes read) [ 4.328723] random: systemd: uninitialized urandom read (16 bytes read) [ 4.331315] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 4.335815] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 4.342134] systemd[1]: Reached target Local Encrypted Volumes. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Timers. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Slices. Starting Setup Virtual Console... [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Sockets. Starting Journal Service... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.962248] device-mapper: uevent: version 1.0.3 [ 4.964907] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.635525] random: fast init done Starting dracut initqueue hook... [ 5.659283] virtio_net virtio0 ens2: renamed from eth0 [ 5.732968] scsi host0: ata_piix [ 5.734969] scsi host1: ata_piix [ 5.737943] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.740486] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.971269] dracut-initqueue[583]: RTNETLINK answers: File exists [ 10.585777] random: crng init done [ 10.587247] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.964503] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.184238] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.501093] SELinux: Disabled at runtime. [ 12.560162] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.567992] systemd[1]: Detected virtualization kvm. [ 12.569644] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 13.167844] systemd[1]: initrd-switch-root.service: Succeeded. [ 13.171334] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 13.177239] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 13.184284] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 13.188149] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 13.195730] systemd[1]: Starting Journal Service... Starting Journal Service... [ 13.200299] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on Process Core Dump Socket. Mounting POSIX Message Queue File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-serial\x2dgetty.slice. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-getty.slice. Mounting Kernel Debug File System... [ OK ] Listening on udev Control Socket. Starting Remount Root and Kernel File Systems... [ OK ] Reached target rpc_pipefs.target. [ OK ] Stopped target Switch Root. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on udev Kernel Soc[ 13.315067] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS ket. Starting udev Coldplug all Devices... [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Created slice User and Session Slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. Mounting Huge Pages File System... Starting Apply Kernel Variables... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Reached target Slices. [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 13.690228] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 14.021783] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 14.043883] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 14.186749] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 14.202780] EDAC sbridge: Ver: 1.1.2 [ 16.328950] Key type dns_resolver registered [ 16.642770] NFS: Registering the id_resolver key type [ 16.644586] Key type id_resolver registered [ 16.646203] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Rebuild Dynamic Linker Cache... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started RPC Bind. [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg443-server login: [ 53.977956] libcfs: loading out-of-tree module taints kernel. [ 54.062973] Key type ._llcrypt registered [ 54.064531] Key type .llcrypt registered [ 54.245128] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 71.649055] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 72.777233] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 72.788216] alg: No test for adler32 (adler32-zlib) [ 74.014553] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 74.799711] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 76.607312] Key type lgssc registered [ 78.511297] Lustre: Echo OBD driver; http://www.lustre.org/ [ 82.706666] hrtimer: interrupt took 7653340 ns [ 93.802514] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 95.147455] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 102.619443] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 107.947418] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 113.730979] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 127.049154] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 140.042812] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 140.115947] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 140.130578] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 141.349206] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 141.389413] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 141.470937] Lustre: lustre-MDT0000: new disk, initializing [ 141.545147] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 141.560311] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 144.801903] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 158.287485] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 158.381841] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 158.481253] Lustre: 6503:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 158.526105] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 158.536685] Lustre: Skipped 1 previous similar message [ 158.682791] Lustre: lustre-MDT0001: new disk, initializing [ 158.832950] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 158.895827] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 158.903172] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 163.076257] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 167.633560] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 175.941854] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 175.992988] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 176.164398] Lustre: lustre-OST0000: new disk, initializing [ 176.169994] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 176.234066] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 180.597632] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 184.878181] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 184.885278] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 184.948659] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 192.772652] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 192.849230] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 192.945640] Lustre: lustre-OST0001: Not available for connect from 0@lo (not set up) [ 192.955163] Lustre: lustre-OST0001: new disk, initializing [ 192.962649] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 193.014164] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 198.141482] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 198.167239] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 198.260566] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 198.305563] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 210.139791] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 215.227966] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 225.216316] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing check_logdir /tmp/testlogs/ [ 229.313518] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing yml_node [ 233.214568] Lustre: DEBUG MARKER: Client: 2.16.61.2 [ 235.577314] Lustre: DEBUG MARKER: MDS: 2.16.61.2 [ 238.189357] Lustre: DEBUG MARKER: OSS: 2.16.61.2 [ 239.610550] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri Nov 14 15:52:09 EST 2025 [ 255.614814] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 256.908103] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 271.328958] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 271.335771] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 271.365319] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 274.912048] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 274.912484] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 274.943914] Lustre: Skipped 3 previous similar messages [ 277.603559] LustreError: 12215:0:(obd_class.h:479:obd_check_dev()) Device 26 not setup [ 277.748789] Lustre: server umount lustre-MDT0000 complete [ 280.967982] LustreError: 6497:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763153572 with bad export cookie 6492181881715053287 [ 280.969423] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 280.980649] LustreError: 6497:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 285.155061] LustreError: 6512:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 285.164122] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 285.176395] LustreError: 6512:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 285.185473] Lustre: Skipped 2 previous similar messages [ 285.185898] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 290.271915] LustreError: 6510:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 290.271981] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 290.313277] Lustre: Skipped 2 previous similar messages [ 291.302155] LustreError: 10450:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 291.327979] LustreError: 10450:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 295.391451] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 295.532520] LustreError: 12416:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 295.548076] LustreError: 12416:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 295.763332] Lustre: server umount lustre-MDT0001 complete [ 299.363575] LustreError: 12619:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 299.380653] LustreError: 12619:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 299.473622] Lustre: server umount lustre-OST0000 complete [ 302.867493] LustreError: 12819:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 302.886099] LustreError: 12819:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 303.029732] Lustre: server umount lustre-OST0001 complete [ 308.650948] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 316.469959] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 324.724975] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 330.621742] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 336.079388] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 341.681221] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 347.452808] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 347.544383] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 347.731876] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 347.783591] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 347.848846] Lustre: lustre-MDT0000: new disk, initializing [ 347.981677] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 348.013783] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 351.221614] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 359.963404] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 360.025632] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 360.132184] Lustre: 15003:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 360.196953] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 360.200812] Lustre: Skipped 1 previous similar message [ 360.260984] Lustre: lustre-MDT0001: new disk, initializing [ 360.357492] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 360.389043] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 360.399060] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 363.326343] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 370.427602] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 373.651578] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 379.542161] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 379.627485] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 379.908846] Lustre: lustre-OST0000: new disk, initializing [ 379.914859] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 380.020708] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 381.285780] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 381.295987] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 381.396981] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 384.766967] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 391.345214] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 394.924397] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 395.147556] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 398.730424] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 398.968740] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 401.889759] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 401.890944] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 401.903611] Lustre: Skipped 1 previous similar message [ 401.912617] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 401.912625] Lustre: Skipped 2 previous similar messages [ 415.199197] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 415.251120] LustreError: 17946:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 415.260769] LustreError: 17946:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 415.337754] Lustre: server umount lustre-OST0000 complete [ 421.862785] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 421.888559] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 421.907787] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 421.913201] Lustre: Skipped 5 previous similar messages [ 424.581962] LustreError: 18148:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 424.594082] LustreError: 18148:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 424.690068] Lustre: server umount lustre-MDT0000 complete [ 427.489657] LustreError: 15016:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 427.505347] LustreError: 15016:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 428.010908] LustreError: 14996:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763153719 with bad export cookie 6492181881715054820 [ 428.012194] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 428.030134] LustreError: 14996:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 428.375845] Lustre: server umount lustre-MDT0001 complete [ 432.808437] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 15:55:22 (1763153722) [ 438.203716] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 438.536762] LustreError: 18876:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 438.590233] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 442.054728] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 448.034860] LustreError: 18877:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 448.063812] LustreError: 18877:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 449.977608] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 454.361617] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 459.599448] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 462.831540] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 468.589502] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 473.131356] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 477.524591] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 491.383297] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 496.611703] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 496.617101] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 496.623761] Lustre: Skipped 1 previous similar message [ 496.627290] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 496.629535] Lustre: Skipped 2 previous similar messages [ 502.686446] LustreError: 21200:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 502.694251] LustreError: 21200:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 502.807659] Lustre: server umount lustre-MDT0000 complete [ 503.777114] LustreError: 18876:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 503.790471] LustreError: 18876:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 505.620332] LustreError: 18857:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763153797 with bad export cookie 6492181881715055702 [ 505.621309] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 505.629802] LustreError: 18857:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 505.906550] Lustre: server umount lustre-MDT0001 complete [ 519.353290] Lustre: server umount lustre-OST0000 complete [ 535.755650] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 544.352152] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 544.693666] LustreError: 22930:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 544.767934] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 544.772258] Lustre: Skipped 2 previous similar messages [ 548.143753] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 556.126956] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 560.012195] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 562.510537] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 568.262321] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 573.927637] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 581.853322] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 581.958690] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 582.134881] Lustre: lustre-OST0001: new disk, initializing [ 582.146953] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 584.027456] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 584.041514] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 584.168864] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 587.369186] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 595.671706] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 598.303252] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 612.825192] LustreError: 26408:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 612.829171] LustreError: 26408:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 612.836923] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 612.851996] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 612.876811] Lustre: Skipped 2 previous similar messages [ 612.880693] LustreError: 24409:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 612.889187] LustreError: 24409:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 612.932173] Lustre: server umount lustre-MDT0000 complete [ 616.093043] LustreError: 22909:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763153907 with bad export cookie 6492181881715056549 [ 616.101031] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 616.113368] LustreError: 22909:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 618.531734] Lustre: server umount lustre-MDT0001 complete [ 632.487736] Lustre: server umount lustre-OST0000 complete [ 646.363448] Lustre: server umount lustre-OST0001 complete [ 659.904477] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 668.839299] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 672.925678] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 680.793520] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 681.154049] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 681.157017] Lustre: Skipped 4 previous similar messages [ 684.790708] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 687.890488] Lustre: 29306:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 694.239903] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 699.726756] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 708.400805] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 714.052499] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 720.532579] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 724.116689] Lustre: 31159:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 735.202963] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 735.214021] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 735.215319] Lustre: Skipped 5 previous similar messages [ 735.239540] Lustre: Skipped 6 previous similar messages [ 740.866211] LustreError: 31360:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 740.871444] LustreError: 31360:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 741.018718] Lustre: server umount lustre-MDT0000 complete [ 744.321473] LustreError: 28169:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763154035 with bad export cookie 6492181881715057844 [ 744.337650] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 744.709746] Lustre: server umount lustre-MDT0001 complete [ 757.873211] Lustre: server umount lustre-OST0000 complete [ 760.806389] Lustre: 3646:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1763154036/real 1763154036] req@ffff98573159bb80 x1848800304188544/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1763154052 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 761.857943] Lustre: server umount lustre-OST0001 complete [ 770.347726] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 16:00:59 (1763154059) [ 793.830607] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 802.385997] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 802.645365] LustreError: 33955:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 802.670065] LustreError: 33955:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [ 805.842649] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 811.904431] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 814.965957] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 817.051297] Lustre: 35071:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 821.995526] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 826.427707] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 833.255479] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 837.649152] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 844.084250] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 846.779753] Lustre: 36926:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 858.089557] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 858.094861] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 858.106409] Lustre: Skipped 2 previous similar messages [ 861.867898] Lustre: server umount lustre-MDT0000 complete [ 865.233717] LustreError: 34689:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763154156 with bad export cookie 6492181881715058943 [ 865.236848] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 865.252257] LustreError: 34689:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 865.671490] Lustre: server umount lustre-MDT0001 complete [ 878.678397] LustreError: 37480:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 878.737418] LustreError: 37480:0:(obd_class.h:479:obd_check_dev()) Skipped 41 previous similar messages [ 878.802965] Lustre: server umount lustre-OST0000 complete [ 892.281351] Lustre: server umount lustre-OST0001 complete [ 901.460173] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 16:03:10 (1763154190) [ 916.452057] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 927.163445] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 931.335636] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 939.944567] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 940.385380] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 940.389562] Lustre: Skipped 7 previous similar messages [ 944.223591] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 946.790763] Lustre: 40171:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 953.905575] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 959.841728] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 967.254603] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 972.802571] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 979.978679] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 983.636619] Lustre: 42025:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 989.658955] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 989.662780] Lustre: Skipped 1 previous similar message [ 992.744965] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 992.747839] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 992.760280] Lustre: Skipped 4 previous similar messages [ 992.766235] Lustre: Skipped 9 previous similar messages [ 998.734385] Lustre: server umount lustre-MDT0000 complete [ 1002.430217] LustreError: 39033:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763154293 with bad export cookie 6492181881715060056 [ 1002.438693] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1002.446891] LustreError: 39033:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1008.842724] Lustre: server umount lustre-MDT0001 complete [ 1016.350933] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1031.350431] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1038.342429] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1040.175807] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1045.485041] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1046.022043] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1046.038142] Lustre: Skipped 1 previous similar message [ 1052.017581] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1054.022369] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1057.663961] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1059.744447] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1066.465503] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1071.325422] Lustre: server umount lustre-MDT0000 complete [ 1071.583784] LustreError: 42906:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1071.605118] LustreError: 42906:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 33 previous similar messages [ 1075.315720] LustreError: 39032:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763154366 with bad export cookie 6492181881715061246 [ 1075.323475] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1075.335740] LustreError: 39032:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1075.613308] Lustre: server umount lustre-MDT0001 complete [ 1090.441993] Lustre: server umount lustre-OST0000 complete [ 1093.629497] Lustre: 3648:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1763154368/real 1763154368] req@ffff98571ff93480 x1848800304322304/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1763154384 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1094.332867] Lustre: server umount lustre-OST0001 complete [ 1103.902624] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 16:06:33 (1763154393) [ 1118.929806] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 1128.794414] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1132.902694] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1141.029597] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1145.204235] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1147.818939] Lustre: 47516:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1153.468163] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1158.616739] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1166.810571] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1172.644203] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1180.156263] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1189.420451] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1202.336773] LustreError: 49593:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 1202.339814] LustreError: 49593:0:(obd_class.h:479:obd_check_dev()) Skipped 51 previous similar messages [ 1202.439572] Lustre: server umount lustre-MDT0000 complete [ 1202.655911] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1205.611112] LustreError: 47902:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763154497 with bad export cookie 6492181881715062023 [ 1205.617163] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1205.619859] LustreError: 47902:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1205.917922] Lustre: server umount lustre-MDT0001 complete [ 1219.450550] Lustre: server umount lustre-OST0000 complete [ 1231.740748] Lustre: server umount lustre-OST0001 complete [ 1240.244865] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 16:08:49 (1763154529) [ 1242.551851] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1244.295719] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 16:08:53 (1763154533) [ 1253.027330] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 1259.584057] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 1268.219785] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1273.582863] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1279.525024] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1279.581392] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1279.823536] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1279.936683] Lustre: lustre-MDT0000: new disk, initializing [ 1280.035587] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1283.874873] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1291.395228] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1298.442918] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1298.524866] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1298.718624] Lustre: lustre-OST0000: new disk, initializing [ 1298.723392] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1298.734421] Lustre: Skipped 1 previous similar message [ 1300.594154] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1300.601622] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1300.654809] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1304.335855] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1312.185485] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1314.928657] Lustre: DEBUG MARKER: create test files [ 1346.529956] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1346.551074] Lustre: Skipped 13 previous similar messages [ 1346.558895] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1346.559336] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1346.581630] Lustre: Skipped 14 previous similar messages [ 1352.254691] Lustre: server umount lustre-OST0000 complete [ 1356.551934] Lustre: server umount lustre-MDT0000 complete [ 1367.263916] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 1370.307173] Key type lgssc unregistered [ 1370.596928] LNet: 54935:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1370.608131] LNetError: 54935:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1370.624846] LNet: Removed LNI 192.168.204.143@tcp [ 1371.682331] Key type .llcrypt unregistered [ 1371.684062] Key type ._llcrypt unregistered [ 1399.967399] Key type ._llcrypt registered [ 1399.970893] Key type .llcrypt registered [ 1400.093690] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 1401.603887] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1401.617554] alg: No test for adler32 (adler32-zlib) [ 1402.682820] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 1402.893877] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 1404.591222] Key type lgssc registered [ 1405.873218] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1413.998303] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1414.025309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1415.521029] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1419.215850] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1425.772270] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1433.069305] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1433.412385] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1439.503357] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1439.613759] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:97 to 0x240000400:129) [ 1439.877345] Lustre: 57539:0:(osd_internal.h:1472:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1439.888534] Lustre: 57539:0:(osd_handler.c:2082:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1439.907188] Lustre: 57539:0:(osd_handler.c:2089:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1439.916310] Lustre: 57539:0:(osd_handler.c:2099:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1439.920590] Lustre: 57539:0:(osd_handler.c:2106:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1439.930481] Lustre: 57539:0:(osd_handler.c:2113:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1446.542722] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1452.813869] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1458.911380] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1490.912186] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1490.921426] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1490.950269] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1496.033790] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1496.892642] LustreError: 58308:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1497.025571] Lustre: server umount lustre-OST0000 complete [ 1501.170737] LustreError: 58510:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1501.183243] LustreError: 58510:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1501.391502] Lustre: server umount lustre-MDT0000 complete [ 1513.551111] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 1516.281651] Key type lgssc unregistered [ 1516.597633] LNet: 59041:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1516.611741] LNetError: 59041:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1516.630902] LNet: Removed LNI 192.168.204.143@tcp [ 1517.694551] Key type .llcrypt unregistered [ 1517.699556] Key type ._llcrypt unregistered [ 1550.548146] Key type ._llcrypt registered [ 1550.549943] Key type .llcrypt registered [ 1550.702404] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 1551.746126] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1551.753990] alg: No test for adler32 (adler32-zlib) [ 1552.927870] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 1553.143187] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 1554.807223] Key type lgssc registered [ 1556.043725] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1565.650770] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1565.689428] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1567.433138] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1572.323553] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1578.963144] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1587.579538] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1587.853953] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1592.004936] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1592.186954] Lustre: 61744:0:(osd_internal.h:1472:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1592.205881] Lustre: 61744:0:(osd_handler.c:2082:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1592.217642] Lustre: 61744:0:(osd_handler.c:2089:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1592.230446] Lustre: 61744:0:(osd_handler.c:2099:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1592.247181] Lustre: 61744:0:(osd_handler.c:2106:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1592.257613] Lustre: 61744:0:(osd_handler.c:2113:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1595.358462] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1602.835125] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1609.289868] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1617.888451] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1617.919949] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1623.015411] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1623.487142] LustreError: 62440:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1623.606888] Lustre: server umount lustre-OST0000 complete [ 1627.525364] LustreError: 62643:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1627.532904] LustreError: 62643:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1627.794936] Lustre: server umount lustre-MDT0000 complete [ 1638.650762] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 1641.997581] Key type lgssc unregistered [ 1642.327188] LNet: 63175:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1642.338655] LNetError: 63175:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1642.359469] LNet: Removed LNI 192.168.204.143@tcp [ 1643.110596] Key type .llcrypt unregistered [ 1643.115833] Key type ._llcrypt unregistered [ 1660.385490] Key type ._llcrypt registered [ 1660.387872] Key type .llcrypt registered [ 1660.501062] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 1674.801604] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 1676.047109] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1676.053590] alg: No test for adler32 (adler32-zlib) [ 1677.132319] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 1677.310289] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 1678.991969] Key type lgssc registered [ 1680.227329] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1689.918977] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1696.522269] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1704.156094] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1712.097566] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1720.245670] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1720.314737] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1720.332450] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1721.779123] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1721.834731] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1721.956494] Lustre: lustre-MDT0000: new disk, initializing [ 1722.140874] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1722.164969] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1726.276950] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1737.409452] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1737.489092] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1737.615554] Lustre: 66305:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1737.661870] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1737.664712] Lustre: Skipped 1 previous similar message [ 1737.722725] Lustre: lustre-MDT0001: new disk, initializing [ 1737.876623] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1737.928891] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1737.950455] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1741.792659] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1749.833623] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1753.613227] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1760.657874] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1760.780324] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1761.099822] Lustre: lustre-OST0000: new disk, initializing [ 1761.103935] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1761.268838] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1767.017132] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1767.028578] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1767.236511] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1767.997868] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1776.525570] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1780.734519] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1781.033106] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1784.709280] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1784.930790] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1787.880827] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1787.883678] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1787.915871] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1787.936641] LustreError: Skipped 1 previous similar message [ 1787.988723] Lustre: Skipped 1 previous similar message [ 1792.769105] LustreError: 69249:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1792.908656] Lustre: server umount lustre-OST0000 complete [ 1798.120123] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1798.130629] Lustre: Skipped 1 previous similar message [ 1798.133549] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1798.143651] Lustre: Skipped 1 previous similar message [ 1802.605330] LustreError: 69450:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1802.610265] LustreError: 69450:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1802.706925] Lustre: server umount lustre-MDT0000 complete [ 1806.678812] LustreError: 66299:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763155098 with bad export cookie 1226973737978270953 [ 1806.680616] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1806.703281] LustreError: 66299:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1806.841377] LustreError: 69652:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1806.845639] LustreError: 69652:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1806.961309] Lustre: server umount lustre-MDT0001 complete [ 1817.456358] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 16:18:26 (1763155106) [ 1829.321368] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 1845.041493] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1870.659197] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 1880.358994] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 1891.704601] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1899.282582] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1906.433417] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1914.053798] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1920.599985] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1920.664364] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1920.847536] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1920.886967] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1920.950469] Lustre: lustre-MDT0000: new disk, initializing [ 1921.032229] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1921.046951] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1924.486958] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1934.993215] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1935.103407] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1935.202317] Lustre: 73891:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1935.283193] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1935.291134] Lustre: Skipped 1 previous similar message [ 1935.368132] Lustre: lustre-MDT0001: new disk, initializing [ 1935.448765] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1935.512553] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1935.533304] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1939.642633] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1947.267890] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1950.816415] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1956.988128] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1957.050392] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1957.250500] Lustre: lustre-OST0000: new disk, initializing [ 1957.258878] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1957.327045] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1962.151131] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 1965.665296] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1965.676971] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1965.752930] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1969.464832] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1973.588502] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1973.866870] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1977.819038] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1978.078353] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1981.415358] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1981.416254] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1981.425516] LustreError: Skipped 1 previous similar message [ 1981.430758] Lustre: Skipped 1 previous similar message [ 1981.433905] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1981.444134] Lustre: Skipped 1 previous similar message [ 1986.528711] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1986.535387] Lustre: Skipped 1 previous similar message [ 1991.652847] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1991.655188] Lustre: Skipped 1 previous similar message [ 1994.209235] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1994.243117] LustreError: 76833:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1994.254091] LustreError: 76833:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1994.413100] Lustre: server umount lustre-OST0000 complete [ 2001.889261] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2001.891353] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2001.892178] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2001.895548] Lustre: Skipped 2 previous similar messages [ 2001.917799] Lustre: Skipped 1 previous similar message [ 2004.634147] LustreError: 77037:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2004.637063] LustreError: 77037:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2004.712713] Lustre: server umount lustre-MDT0000 complete [ 2007.011646] LustreError: 73903:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2007.042107] LustreError: 73903:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 2008.677787] LustreError: 73884:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763155300 with bad export cookie 1226973737978271828 [ 2008.680517] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2008.706719] LustreError: 73884:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2009.046918] Lustre: server umount lustre-MDT0001 complete [ 2018.808833] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 16:21:48 (1763155308) [ 2025.365350] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2026.042694] LustreError: 77909:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2026.135493] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2031.352894] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2031.586241] LustreError: 77909:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2035.684309] LustreError: 77909:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2039.349573] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2043.167866] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2048.356344] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2052.634566] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2060.581626] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2061.006744] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2061.016028] Lustre: Skipped 1 previous similar message [ 2066.657790] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2072.253451] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2075.997727] Lustre: 80248:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting c5caf560-c0d4-46de-b822-1ce3e2d1253e at adminstrative request [ 2076.646629] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2076.694167] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2086.890620] Lustre: MGS: Client c5caf560-c0d4-46de-b822-1ce3e2d1253e (at 0@lo) reconnecting [ 2086.900354] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x1107161d28ed9bcd to 0x1107161d28ed9e6d [ 2086.919982] Lustre: MGC192.168.204.143@tcp: Connection restored to 0@lo (at 0@lo) [ 2112.373528] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2112.460491] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2112.569726] Lustre: lustre-OST0001: new disk, initializing [ 2112.573402] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2117.739951] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2121.336896] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2121.348487] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2121.460132] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2124.059477] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2126.823606] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2126.825742] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2126.830367] LustreError: Skipped 1 previous similar message [ 2126.832553] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2126.836826] Lustre: Skipped 1 previous similar message [ 2126.845536] Lustre: Skipped 1 previous similar message [ 2128.510172] LustreError: 81386:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 2128.553548] LustreError: 81386:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 2128.620119] Lustre: server umount lustre-OST0001 complete [ 2131.941322] LustreError: 79508:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2131.953416] LustreError: 79508:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2132.966198] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2136.036627] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2136.053154] Lustre: Skipped 1 previous similar message [ 2138.656355] Lustre: server umount lustre-OST0000 complete [ 2148.540773] LustreError: 81790:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 2148.554631] LustreError: 81790:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 2148.787714] Lustre: server umount lustre-MDT0000 complete [ 2152.403393] LustreError: 80643:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763155443 with bad export cookie 1226973737978273389 [ 2152.407373] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2152.409549] LustreError: 80643:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2152.662873] Lustre: server umount lustre-MDT0001 complete [ 2159.766692] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 16:24:09 (1763155449) [ 2161.189219] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2162.919734] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 16:24:12 (1763155452) [ 2164.487671] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2166.123726] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 16:24:15 (1763155455) [ 2167.546468] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2168.828680] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 16:24:18 (1763155458) [ 2169.989179] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2173.910673] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2174.834657] loop: module loaded [ 2174.857576] loop0: detected capacity change from 0 to 4294967296 [ 2174.905490] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2176.547303] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2179.499078] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2182.812928] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2189.259165] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2192.324713] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2195.685600] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2199.293402] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2205.564946] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2205.622357] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2205.645926] LustreError: 84563:0:(obd_config.c:783:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2205.650238] LustreError: 84563:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2205.654017] LustreError: 84563:0:(tgt_mount.c:2447:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2205.658216] LustreError: 84563:0:(super25.c:199:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2222.285591] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 2231.249041] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 2240.446762] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2246.259794] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2250.876534] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2256.195431] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2262.834272] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2262.886292] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2263.041219] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2263.073049] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2263.139418] Lustre: lustre-MDT0000: new disk, initializing [ 2263.275826] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2263.278789] Lustre: Skipped 1 previous similar message [ 2263.298642] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2267.045717] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2277.336276] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2277.384923] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2277.450990] Lustre: 87406:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2277.765959] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2280.913568] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2287.388964] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2290.866449] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2295.766544] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2295.803957] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2300.315744] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2302.020798] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2302.099025] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2306.299860] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2309.151305] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2309.285066] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2311.816829] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2311.935546] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2317.281552] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2317.287719] Lustre: Skipped 2 previous similar messages [ 2317.292359] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2317.295571] Lustre: Skipped 7 previous similar messages [ 2319.040143] LustreError: 90347:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2319.050750] LustreError: 90347:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2319.116624] Lustre: server umount lustre-OST0000 complete [ 2327.965792] Lustre: server umount lustre-MDT0000 complete [ 2331.662183] LustreError: 89153:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763155623 with bad export cookie 1226973737978273984 [ 2331.665744] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2331.693614] LustreError: 89153:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2332.120210] Lustre: server umount lustre-MDT0001 complete [ 2339.913200] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 16:27:09 (1763155629) [ 2346.072073] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2346.589450] LustreError: 91422:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2346.598748] LustreError: 91422:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2350.247808] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2359.407870] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2363.667888] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2369.229506] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2372.648593] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2379.796732] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2385.726367] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2392.089866] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2396.492645] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2396.791480] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2400.183902] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2400.389381] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2406.956577] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2407.031357] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2407.175161] Lustre: lustre-OST0001: new disk, initializing [ 2407.177535] Lustre: Skipped 2 previous similar messages [ 2407.186870] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2407.189865] Lustre: Skipped 3 previous similar messages [ 2407.289137] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2407.292808] Lustre: Skipped 5 previous similar messages [ 2413.102760] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2415.206836] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2415.213715] Lustre: Skipped 2 previous similar messages [ 2415.217579] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2415.326247] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2419.753647] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2423.717719] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2423.893567] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2427.833219] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2428.135291] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2446.706906] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2471.859611] Lustre: Failing over lustre-MDT0000 [ 2471.906672] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2471.908452] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2471.937073] Lustre: Skipped 5 previous similar messages [ 2471.946630] Lustre: Skipped 6 previous similar messages [ 2474.246113] LustreError: 96332:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 2474.250486] LustreError: 96332:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2474.373814] Lustre: server umount lustre-MDT0000 complete [ 2476.304319] LustreError: 91416:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.204.43@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2476.329890] LustreError: 91416:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2493.400275] Lustre: 64518:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1763155768/real 1763155768] req@0000000076e58dc1 x1848801983977344/t0(0) o400->MGC192.168.204.143@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1763155784 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2493.443119] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2499.042187] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2499.044100] LDISKFS-fs (dm-0): recovery complete [ 2499.049749] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2502.626915] LustreError: 64514:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@0000000076e58dc1 x1848801983985024/t0(0) o250->MGC192.168.204.143@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2503.090112] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2507.688478] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2507.993428] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2507.993562] Lustre: 91417:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2508.277493] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2508.365251] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2508.695238] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2508.715044] Lustre: 97081:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2508.756234] Lustre: 97081:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 7 previous similar messages [ 2509.105645] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2509.115373] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 1 previous similar message [ 2509.440633] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2509.456421] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 1 previous similar message [ 2509.805348] Lustre: 97081:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2509.818067] Lustre: 97081:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 2 previous similar messages [ 2510.136079] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2510.160128] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 2 previous similar messages [ 2510.487174] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2510.495602] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 2 previous similar messages [ 2512.097960] Lustre: 97081:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2512.109775] Lustre: 97081:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 6 previous similar messages [ 2512.435828] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2512.460448] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 6 previous similar messages [ 2512.783342] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2512.798522] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 6 previous similar messages [ 2516.319729] Lustre: 97081:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2516.344525] Lustre: 97081:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 12 previous similar messages [ 2516.664405] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2516.684972] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 12 previous similar messages [ 2517.015589] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2517.023181] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 12 previous similar messages [ 2524.333798] Lustre: 97081:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2524.358501] Lustre: 97081:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 24 previous similar messages [ 2524.692554] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2524.704618] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 24 previous similar messages [ 2525.023511] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2525.030829] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 24 previous similar messages [ 2540.541796] Lustre: 97081:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2540.551992] Lustre: 97081:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 50 previous similar messages [ 2540.872114] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2540.884134] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 50 previous similar messages [ 2541.199440] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2541.203711] LustreError: 97081:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 50 previous similar messages [ 2567.215249] Lustre: 97081:0:(ldlm_lib.c:2375:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2567.232503] Lustre: 97081:0:(ldlm_lib.c:2385:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2567.249138] Lustre: 97081:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 28863555-9e21-4eeb-b48f-20fee4f72ce5@192.168.204.43@tcp [ 2567.274803] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2567.297204] Lustre: 97081:0:(ldlm_lib.c:1897:abort_req_replay_queue()) @@@ aborted: req@000000004f28f2e5 x1848801969118592/t0(8589934781) o101->28863555-9e21-4eeb-b48f-20fee4f72ce5@192.168.204.43@tcp:594/0 lens 592/0 e 0 to 0 dl 1763155869 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2567.317580] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2567.332896] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2567.347206] Lustre: lustre-MDT0000: Denying connection for new client 28863555-9e21-4eeb-b48f-20fee4f72ce5 (at 192.168.204.43@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) already passed deadline 0:00 [ 2567.371285] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2567.379022] Lustre: Skipped 2 previous similar messages [ 2567.379433] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2567.422485] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 2567.427810] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 2583.521339] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2583.541989] LustreError: Skipped 1 previous similar message [ 2583.548924] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2583.584788] Lustre: Skipped 1 previous similar message [ 2588.786733] Lustre: server umount lustre-OST0000 complete [ 2590.184165] LustreError: 93020:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2590.197112] LustreError: 93020:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 39 previous similar messages [ 2598.112345] Lustre: server umount lustre-OST0001 complete [ 2603.488751] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2603.492108] LustreError: Skipped 3 previous similar messages [ 2603.503289] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2603.506077] Lustre: Skipped 9 previous similar messages [ 2607.878940] LustreError: 98148:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 2607.892586] LustreError: 98148:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 2608.182179] Lustre: server umount lustre-MDT0000 complete [ 2611.903761] LustreError: 98149:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763155903 with bad export cookie 1226973737978325196 [ 2611.909119] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2611.979787] LustreError: 98149:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2612.322404] Lustre: server umount lustre-MDT0001 complete [ 2620.825828] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 16:31:50 (1763155910) [ 2628.192895] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2628.197883] Lustre: *** cfs_fail_loc=197, val=0*** [ 2643.809129] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2648.927699] LustreError: 99042:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.143@tcp: failed processing log, type 4: rc = -110 [ 2654.081369] Lustre: *** cfs_fail_loc=197, val=0*** [ 2654.082959] Lustre: Skipped 65 previous similar messages [ 2674.655383] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2674.661651] Lustre: Skipped 1 previous similar message [ 2680.633377] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2685.433781] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2687.211991] Lustre: server umount lustre-OST0000 complete [ 2694.510956] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 16:33:04 (1763155984) [ 2713.365476] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2713.690183] LustreError: 100552:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2713.708842] LustreError: 100552:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 2716.861087] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2723.817787] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2727.765800] LustreError: 100583:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2727.778713] LustreError: 100583:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2727.792268] LustreError: 100583:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 14, retries 0, failed: rc = -5 [ 2728.161221] Lustre: server umount lustre-MDT0000 complete [ 2743.443910] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2758.539385] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 2768.023047] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 2776.010093] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2781.706231] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2787.106530] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2793.174993] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2799.253315] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2799.298041] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2799.445888] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2799.487333] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2799.542459] Lustre: lustre-MDT0000: new disk, initializing [ 2799.620621] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2802.688549] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2811.790600] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2811.848732] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2811.913283] Lustre: 104625:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2812.125114] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2815.099754] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2821.688807] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2825.009121] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2830.937366] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2830.992243] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2835.822912] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2838.168708] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2842.458626] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2846.498447] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2846.688414] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2849.722635] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2849.888921] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2853.344839] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2853.354251] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2853.363152] Lustre: Skipped 5 previous similar messages [ 2857.488300] Lustre: server umount lustre-OST0000 complete [ 2863.590844] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2863.595532] Lustre: Skipped 4 previous similar messages [ 2874.335374] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2874.404359] LustreError: 107764:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2874.408807] LustreError: 107764:0:(obd_class.h:479:obd_check_dev()) Skipped 37 previous similar messages [ 2874.455773] Lustre: server umount lustre-MDT0000 complete [ 2876.834205] LustreError: 106377:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763156168 with bad export cookie 1226973737978335878 [ 2876.839228] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2876.841675] LustreError: 106377:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2877.146971] Lustre: server umount lustre-MDT0001 complete [ 2883.629372] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 16:36:13 (1763156173) [ 2889.874363] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 2891.601547] Key type lgssc unregistered [ 2891.812563] LNet: 108694:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2891.821806] LNetError: 108694:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2891.836710] LNet: Removed LNI 192.168.204.143@tcp [ 2892.354170] Key type .llcrypt unregistered [ 2892.356061] Key type ._llcrypt unregistered [ 2904.464161] Key type ._llcrypt registered [ 2904.465997] Key type .llcrypt registered [ 2904.544487] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 2915.051235] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 2915.822128] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2915.839411] alg: No test for adler32 (adler32-zlib) [ 2916.814436] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 2916.999323] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 2918.647183] Key type lgssc registered [ 2919.342351] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2926.007572] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2931.722890] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2937.509168] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2943.083527] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2947.524867] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2952.275482] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2952.322859] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2952.332533] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2953.530367] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2953.576438] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2953.659599] Lustre: lustre-MDT0000: new disk, initializing [ 2953.779459] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2953.801676] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2957.192742] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2965.175332] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2965.236323] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2965.402095] Lustre: lustre-OST0000: new disk, initializing [ 2965.404611] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2965.414251] Lustre: Skipped 1 previous similar message [ 2965.494510] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2965.844050] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2965.849145] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2965.878831] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2970.164902] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2979.511817] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2979.562617] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2979.665274] Lustre: lustre-OST0001: new disk, initializing [ 2979.671947] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2979.735722] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2984.770713] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 2988.625455] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2988.631206] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2988.657575] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2993.640473] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2993.645169] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2993.647051] Lustre: Skipped 1 previous similar message [ 2997.597688] LustreError: 114250:0:(obd_class.h:479:obd_check_dev()) Device 17 not setup [ 2997.738289] Lustre: server umount lustre-MDT0000 complete [ 3002.743906] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3005.927256] LustreError: 112009:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763156297 with bad export cookie 14493388379699330062 [ 3005.932583] LustreError: 112009:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3005.932961] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3005.970610] LustreError: 114845:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 3005.974820] LustreError: 114845:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3006.007141] Lustre: server umount lustre-OST0000 complete [ 3008.288719] LustreError: 115045:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 3008.291953] LustreError: 115045:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 3008.383358] Lustre: server umount lustre-OST0001 complete [ 3013.974055] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 16:38:23 (1763156303) [ 3019.737574] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 3024.326336] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3024.860845] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3028.160674] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3028.787965] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3033.337532] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 16:38:43 (1763156323) [ 3042.058284] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3043.339570] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3044.632333] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3045.916274] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3046.610377] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3047.939037] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3058.694371] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 3065.636790] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 3072.457148] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3076.921134] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3081.278738] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3085.373083] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3090.133297] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3090.200757] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3090.347044] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3090.372328] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3090.423454] Lustre: lustre-MDT0000: new disk, initializing [ 3090.492252] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3090.510419] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3093.033497] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3100.271206] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3100.320362] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3100.396732] Lustre: 120374:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3100.431167] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3100.435393] Lustre: Skipped 1 previous similar message [ 3100.497333] Lustre: lustre-MDT0001: new disk, initializing [ 3100.565483] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3100.592936] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3100.602641] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3103.203632] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3107.747780] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3109.865510] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3114.102656] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3114.150434] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3114.309354] Lustre: lustre-OST0000: new disk, initializing [ 3114.312277] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3114.371340] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3118.062609] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3119.636710] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3119.643464] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3119.718912] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3122.896515] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3125.022967] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3125.142638] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3127.420991] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3127.571714] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3129.829142] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3129.834296] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3129.843133] LustreError: Skipped 1 previous similar message [ 3129.844162] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3129.844166] Lustre: Skipped 1 previous similar message [ 3129.860329] Lustre: Skipped 1 previous similar message [ 3134.949932] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3134.954726] Lustre: Skipped 1 previous similar message [ 3140.067347] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3140.069316] Lustre: Skipped 1 previous similar message [ 3143.135136] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3143.167159] LustreError: 123314:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3143.174373] LustreError: 123314:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3143.228247] Lustre: server umount lustre-OST0000 complete [ 3146.723924] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3146.727235] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3146.734788] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3146.737777] Lustre: Skipped 1 previous similar message [ 3150.307625] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3151.509854] LustreError: 123517:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 3151.513351] LustreError: 123517:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3151.564064] Lustre: server umount lustre-MDT0000 complete [ 3154.062412] LustreError: 121156:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763156445 with bad export cookie 14493388379699331280 [ 3154.066501] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3154.078857] LustreError: 121156:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3154.299986] Lustre: server umount lustre-MDT0001 complete [ 3159.238805] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 16:40:49 (1763156449) [ 3163.096801] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3163.405290] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3165.559695] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3168.737084] LustreError: 124388:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3168.750036] LustreError: 124388:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3171.106511] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3173.886047] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3177.874615] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3180.652427] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3185.941797] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3190.459440] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3194.725586] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3201.547478] Lustre: *** cfs_fail_loc=159, val=0*** [ 3217.140473] Lustre: lustre-MDT0000: Client e25200f7-44b4-4093-b3e4-81506d425ff3 (at 192.168.204.43@tcp) reconnecting [ 3218.271019] Lustre: *** cfs_fail_loc=159, val=0*** [ 3218.272942] Lustre: Skipped 5 previous similar messages [ 3234.539299] Lustre: lustre-MDT0000: Client e25200f7-44b4-4093-b3e4-81506d425ff3 (at 192.168.204.43@tcp) reconnecting [ 3250.145342] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3250.150886] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3250.160305] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3250.163239] Lustre: Skipped 2 previous similar messages [ 3253.219486] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3256.054220] LustreError: 126881:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3256.062943] LustreError: 126881:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 3256.162287] Lustre: server umount lustre-OST0000 complete [ 3258.340199] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3258.340373] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3258.352312] Lustre: Skipped 2 previous similar messages [ 3264.450167] Lustre: server umount lustre-MDT0000 complete [ 3266.835543] LustreError: 126779:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763156558 with bad export cookie 14493388379699332162 [ 3266.836728] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3266.846498] LustreError: 126779:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3267.108864] Lustre: server umount lustre-MDT0001 complete [ 3272.915234] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 3274.771488] Key type lgssc unregistered [ 3274.965655] LNet: 127817:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3274.970533] LNetError: 127817:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3274.977582] LNet: Removed LNI 192.168.204.143@tcp [ 3275.480712] Key type .llcrypt unregistered [ 3275.482374] Key type ._llcrypt unregistered [ 3289.922688] Key type ._llcrypt registered [ 3289.925352] Key type .llcrypt registered [ 3289.990565] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 3290.480229] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3290.528776] alg: No test for adler32 (adler32-zlib) [ 3291.474652] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 3291.612622] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 3293.239220] Key type lgssc registered [ 3293.992255] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3299.296168] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3299.311461] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3300.714357] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3303.702888] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3310.600743] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3310.935681] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3314.421284] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3319.343272] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3322.240419] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3328.105309] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3328.408163] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3333.136826] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3337.399647] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3346.929408] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3355.252823] Lustre: *** cfs_fail_loc=159, val=0*** [ 3371.763792] Lustre: lustre-MDT0000: Client 22961d09-35c7-481d-a5d5-160ed7dd75b1 (at 192.168.204.43@tcp) reconnecting [ 3373.582161] Lustre: *** cfs_fail_loc=159, val=0*** [ 3389.192596] Lustre: lustre-MDT0001: Client 22961d09-35c7-481d-a5d5-160ed7dd75b1 (at 192.168.204.43@tcp) reconnecting [ 3390.604209] Lustre: *** cfs_fail_loc=159, val=0*** [ 3390.606443] Lustre: Skipped 6 previous similar messages [ 3407.092487] Lustre: lustre-MDT0001: Client 22961d09-35c7-481d-a5d5-160ed7dd75b1 (at 192.168.204.43@tcp) reconnecting [ 3422.397462] Lustre: 131934:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3424.678303] Lustre: *** cfs_fail_loc=159, val=0*** [ 3424.684318] Lustre: Skipped 1 previous similar message [ 3440.376231] Lustre: lustre-MDT0000: Client c92b43a7-e120-47f7-b655-f32d768e6809 (at 192.168.204.43@tcp) reconnecting [ 3441.583731] Lustre: *** cfs_fail_loc=159, val=0*** [ 3441.585670] Lustre: Skipped 14 previous similar messages [ 3457.255658] Lustre: lustre-MDT0000: Client c92b43a7-e120-47f7-b655-f32d768e6809 (at 192.168.204.43@tcp) reconnecting [ 3458.192122] Lustre: 132226:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3474.919664] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3474.919708] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3474.938751] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3474.942064] Lustre: Skipped 1 previous similar message [ 3480.033814] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3480.039112] Lustre: Skipped 1 previous similar message [ 3480.373107] LustreError: 132358:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3480.434615] Lustre: server umount lustre-OST0000 complete [ 3485.152816] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3485.153969] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3485.163435] Lustre: Skipped 2 previous similar messages [ 3485.166484] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3488.854699] LustreError: 132562:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3488.859108] LustreError: 132562:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3488.971754] Lustre: server umount lustre-MDT0000 complete [ 3490.273610] LustreError: 129974:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3490.282231] LustreError: 129974:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3491.359440] LustreError: 129169:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763156782 with bad export cookie 6008189367075902571 [ 3491.360663] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3491.367645] LustreError: 129169:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3491.467250] LustreError: 132763:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3491.469550] LustreError: 132763:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3491.591238] Lustre: server umount lustre-MDT0001 complete [ 3497.553281] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 3499.360636] Key type lgssc unregistered [ 3499.568746] LNet: 133294:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3499.574781] LNetError: 133294:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3500.582822] LNet: Removed LNI 192.168.204.143@tcp [ 3501.031654] Key type .llcrypt unregistered [ 3501.033954] Key type ._llcrypt unregistered [ 3514.837287] Key type ._llcrypt registered [ 3514.839076] Key type .llcrypt registered [ 3514.904536] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 3515.456939] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3515.531844] alg: No test for adler32 (adler32-zlib) [ 3516.517779] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 3516.687071] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 3518.327183] Key type lgssc registered [ 3518.988551] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3523.953879] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3523.977443] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3525.344426] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3527.890765] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3534.636974] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3534.827191] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 3534.964860] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3538.215635] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3542.373703] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3544.837284] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3549.436126] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3549.652708] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3550.708387] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3553.545346] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3557.043445] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3559.918944] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3565.379400] Lustre: 136915:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3567.116960] Lustre: 136963:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3568.428645] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3569.087031] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3571.581353] Lustre: 137111:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3572.227318] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3572.859904] Lustre: 137209:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3575.270446] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3575.270547] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3575.275347] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3575.282608] Lustre: Skipped 1 previous similar message [ 3575.289206] Lustre: Skipped 1 previous similar message [ 3580.385579] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3580.394331] Lustre: Skipped 1 previous similar message [ 3581.262790] LustreError: 137307:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3581.373496] Lustre: server umount lustre-OST0000 complete [ 3585.508497] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3585.517132] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3585.520051] Lustre: Skipped 1 previous similar message [ 3590.625847] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3590.628928] Lustre: Skipped 1 previous similar message [ 3598.303227] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3598.389844] LustreError: 137515:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3598.393993] LustreError: 137515:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3598.468223] Lustre: server umount lustre-MDT0000 complete [ 3600.864642] LustreError: 134663:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3600.890442] LustreError: 134663:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3601.656641] LustreError: 137013:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763156893 with bad export cookie 10549980522244575151 [ 3601.668583] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3601.678449] LustreError: 137013:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3601.769487] LustreError: 137717:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3601.774084] LustreError: 137717:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3601.929058] Lustre: server umount lustre-MDT0001 complete [ 3611.063348] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 3613.311984] Key type lgssc unregistered [ 3613.593358] LNet: 138249:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3613.605760] LNetError: 138249:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3613.619269] LNet: Removed LNI 192.168.204.143@tcp [ 3614.312136] Key type .llcrypt unregistered [ 3614.313956] Key type ._llcrypt unregistered [ 3631.028553] Key type ._llcrypt registered [ 3631.030261] Key type .llcrypt registered [ 3631.103993] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 3631.668338] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3631.687785] alg: No test for adler32 (adler32-zlib) [ 3632.634839] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 3632.803099] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 3634.423272] Key type lgssc registered [ 3635.165816] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3640.349611] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3640.372652] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3641.710858] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3644.093353] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3649.397092] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3649.678595] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3652.009911] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3655.238924] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3657.224076] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3660.535837] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3660.811527] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3664.181283] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3666.988703] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 3667.722611] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3675.180438] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 3680.930412] Lustre: *** cfs_fail_loc=159, val=0*** [ 3680.932313] Lustre: Skipped 1 previous similar message [ 3696.364663] Lustre: lustre-MDT0000: Client 6d892a12-ed11-40a6-acd2-c2565e9e525d (at 192.168.204.43@tcp) reconnecting [ 3716.067556] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3716.073820] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3716.080122] Lustre: Skipped 1 previous similar message [ 3716.083247] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3716.085996] Lustre: Skipped 1 previous similar message [ 3719.170478] LustreError: 142024:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3719.262366] Lustre: server umount lustre-OST0000 complete [ 3726.306644] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3726.307113] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3726.307802] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3726.316033] Lustre: Skipped 1 previous similar message [ 3727.952804] LustreError: 142225:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3727.955423] LustreError: 142225:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3728.024601] Lustre: server umount lustre-MDT0000 complete [ 3730.330095] LustreError: 139603:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763157021 with bad export cookie 9148119751905721188 [ 3730.335715] LustreError: 139603:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3730.335755] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3730.426359] LustreError: 142426:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3730.430273] LustreError: 142426:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3730.535746] Lustre: server umount lustre-MDT0001 complete [ 3735.893931] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 3737.515048] Key type lgssc unregistered [ 3737.694532] LNet: 142957:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3737.699808] LNetError: 142957:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3737.714340] LNet: Removed LNI 192.168.204.143@tcp [ 3738.208935] Key type .llcrypt unregistered [ 3738.211749] Key type ._llcrypt unregistered [ 3753.916529] Key type ._llcrypt registered [ 3753.918126] Key type .llcrypt registered [ 3753.996942] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 3754.824117] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3754.843185] alg: No test for adler32 (adler32-zlib) [ 3755.822189] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 3755.977868] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 3757.610463] Key type lgssc registered [ 3758.297311] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3763.207486] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3763.223529] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3764.532442] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3766.663977] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3771.640676] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3771.877953] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3774.289021] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3778.190159] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3780.922334] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3784.840271] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3785.075566] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3788.214219] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3789.162745] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 3791.110757] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3797.355799] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 3811.762394] Lustre: lustre-OST0000: evicting 37c5a71d-e645-487d-92f5-d1dc490de918 (at 192.168.204.43@tcp) by administrative request [ 3811.768196] Lustre: Skipped 1 previous similar message [ 3838.435829] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3838.437156] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3838.437369] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3838.441360] Lustre: Skipped 1 previous similar message [ 3843.515144] LustreError: 147119:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3843.592307] Lustre: server umount lustre-OST0000 complete [ 3848.672238] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3848.677536] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3848.679239] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3848.684559] Lustre: Skipped 1 previous similar message [ 3848.688313] Lustre: Skipped 1 previous similar message [ 3852.066996] LustreError: 147321:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3852.071898] LustreError: 147321:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3852.120387] Lustre: server umount lustre-MDT0000 complete [ 3853.794379] LustreError: 144331:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3853.807318] LustreError: 144331:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3854.710417] LustreError: 144313:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763157146 with bad export cookie 15649807529808778047 [ 3854.731989] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3854.848025] LustreError: 147523:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3854.853927] LustreError: 147523:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3855.003401] Lustre: server umount lustre-MDT0001 complete [ 3861.733112] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 3863.638637] Key type lgssc unregistered [ 3863.845660] LNet: 148054:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3863.852233] LNetError: 148054:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3863.870919] LNet: Removed LNI 192.168.204.143@tcp [ 3864.383072] Key type .llcrypt unregistered [ 3864.385393] Key type ._llcrypt unregistered [ 3903.742691] Key type ._llcrypt registered [ 3903.744475] Key type .llcrypt registered [ 3903.803198] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 3904.236261] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3904.246606] alg: No test for adler32 (adler32-zlib) [ 3905.194182] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 3905.304295] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 3906.951199] Key type lgssc registered [ 3907.507477] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3911.625495] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3911.642889] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3912.927317] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3914.925861] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3919.709350] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3919.946508] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3922.031711] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3925.114268] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3927.378718] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3931.484824] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3931.714964] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3934.849385] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 3937.896829] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 3938.428995] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3945.070577] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 3948.513380] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3948.516636] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3948.524946] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3950.560715] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3950.565904] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3950.572255] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3954.360598] LustreError: 152753:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3954.412371] Lustre: server umount lustre-OST0000 complete [ 3960.800749] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3960.802391] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3960.803439] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3960.803444] Lustre: Skipped 1 previous similar message [ 3960.811575] Lustre: Skipped 1 previous similar message [ 3962.513978] LustreError: 152955:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3962.516782] LustreError: 152955:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3962.570246] Lustre: server umount lustre-MDT0000 complete [ 3964.576616] LustreError: 150481:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763157255 with bad export cookie 8842060959266785189 [ 3964.579449] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3964.581768] LustreError: 150481:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3964.658026] LustreError: 153156:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3964.661264] LustreError: 153156:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3964.808159] Lustre: server umount lustre-MDT0001 complete [ 3969.311541] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 3970.596540] Key type lgssc unregistered [ 3970.757838] LNet: 153686:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3970.761620] LNetError: 153686:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3970.774115] LNet: Removed LNI 192.168.204.143@tcp [ 3971.160936] Key type .llcrypt unregistered [ 3971.162720] Key type ._llcrypt unregistered [ 3984.207710] Key type ._llcrypt registered [ 3984.209394] Key type .llcrypt registered [ 3984.270467] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 3984.729929] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3984.800180] alg: No test for adler32 (adler32-zlib) [ 3985.710791] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 3985.836116] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 3987.455186] Key type lgssc registered [ 3988.023442] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3991.607781] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3995.622207] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 16:54:45 (1763157285) [ 4003.365355] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 4007.839202] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 4012.312690] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4015.559296] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4018.362942] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4021.400431] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4025.020993] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4025.050695] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4025.059190] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4026.227106] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4026.267470] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4026.312683] Lustre: lustre-MDT0000: new disk, initializing [ 4026.365854] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4026.381603] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4028.306519] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4034.166530] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4034.207183] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4034.259456] Lustre: 157879:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4034.292496] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4034.295442] Lustre: Skipped 1 previous similar message [ 4034.338949] Lustre: lustre-MDT0001: new disk, initializing [ 4034.391099] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4034.411704] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4034.416666] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4036.162973] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4040.419978] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4042.270555] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4046.122513] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4046.158530] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4046.316117] Lustre: lustre-OST0000: new disk, initializing [ 4046.319286] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4046.377822] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4049.338299] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4049.525782] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4049.532078] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4049.595181] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4054.230181] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4060.133582] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4060.135697] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4060.137182] LustreError: Skipped 1 previous similar message [ 4060.143791] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4060.148130] Lustre: Skipped 1 previous similar message [ 4065.250644] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4065.253636] Lustre: Skipped 1 previous similar message [ 4065.716227] LustreError: 160464:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4065.778143] Lustre: server umount lustre-OST0000 complete [ 4070.368488] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4070.372582] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4070.387331] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4070.397412] Lustre: Skipped 2 previous similar messages [ 4073.802584] LustreError: 160665:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 4073.805158] LustreError: 160665:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4073.864315] Lustre: server umount lustre-MDT0000 complete [ 4076.020863] LustreError: 157869:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763157367 with bad export cookie 17965169338039258614 [ 4076.025490] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4076.034052] LustreError: 157869:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4076.137424] LustreError: 160866:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4076.144683] LustreError: 160866:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4076.279723] Lustre: server umount lustre-MDT0001 complete [ 4082.644551] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 4084.074736] Key type lgssc unregistered [ 4084.230557] LNet: 161399:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4084.235162] LNetError: 161399:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4084.243734] LNet: Removed LNI 192.168.204.143@tcp [ 4084.688118] Key type .llcrypt unregistered [ 4084.693337] Key type ._llcrypt unregistered [ 4097.075159] Key type ._llcrypt registered [ 4097.076990] Key type .llcrypt registered [ 4097.127599] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 4097.518128] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4097.568176] alg: No test for adler32 (adler32-zlib) [ 4098.491478] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 4098.609801] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 4100.215215] Key type lgssc registered [ 4100.731935] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4104.335792] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4104.346463] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4105.584787] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4107.419244] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4111.587905] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4111.762203] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4113.668047] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4116.621321] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4118.643858] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4122.319969] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4122.502958] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4125.211420] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 4127.822662] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4129.721330] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4140.079758] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4140.087730] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4140.096641] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4140.101859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4140.107816] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4140.569732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.032399] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4141.035689] Lustre: Skipped 1 previous similar message [ 4141.043664] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4141.049040] Lustre: Skipped 1 previous similar message [ 4141.052921] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4141.056589] Lustre: Skipped 1 previous similar message [ 4141.059233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.065667] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4141.068718] Lustre: Skipped 1 previous similar message [ 4141.524771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4142.010533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4142.464809] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4142.468016] Lustre: Skipped 2 previous similar messages [ 4142.475902] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4142.481836] Lustre: Skipped 2 previous similar messages [ 4142.485765] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4142.489946] Lustre: Skipped 2 previous similar messages [ 4142.492264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4142.498436] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4142.501332] Lustre: Skipped 2 previous similar messages [ 4142.893524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4143.303628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4143.740549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4144.144180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4144.587394] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4144.590672] Lustre: Skipped 4 previous similar messages [ 4144.605210] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4144.612320] Lustre: Skipped 4 previous similar messages [ 4144.616430] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4144.619581] Lustre: Skipped 4 previous similar messages [ 4144.621787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4144.629773] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4144.635547] Lustre: Skipped 4 previous similar messages [ 4145.057917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4145.448494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4145.840861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4146.252451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4146.647706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.048234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.478931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.873107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.254929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.664916] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4148.668728] Lustre: Skipped 9 previous similar messages [ 4148.676212] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4148.682107] Lustre: Skipped 9 previous similar messages [ 4148.684843] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4148.688213] Lustre: Skipped 9 previous similar messages [ 4148.690581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.696265] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4148.701393] Lustre: Skipped 9 previous similar messages [ 4149.091165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4149.484590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4149.909167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4150.342333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4150.747131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4151.210072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4151.657369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.106728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.558972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.056234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.515783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.976982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4154.479328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4155.001160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4155.426076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4155.862232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4156.307237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4156.750541] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4156.753821] Lustre: Skipped 17 previous similar messages [ 4156.761065] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4156.768232] Lustre: Skipped 17 previous similar messages [ 4156.770814] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4156.773487] Lustre: Skipped 17 previous similar messages [ 4156.777070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4156.785060] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4156.788966] Lustre: Skipped 17 previous similar messages [ 4157.292133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4157.757276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.381426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.902641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4159.408784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4159.873239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4160.344961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4160.782259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4161.244059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4161.703210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4162.188082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4162.694915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4163.163301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4163.798424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4164.344255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4164.847298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4165.451486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4165.886287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4166.346332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4166.816896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4167.283349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4167.746589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4168.189414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4168.619080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4169.086758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4169.525545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4169.950227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4170.419119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4170.853712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.339336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.828086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4172.306112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4172.761493] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4172.764511] Lustre: Skipped 32 previous similar messages [ 4172.775912] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4172.783437] Lustre: Skipped 32 previous similar messages [ 4172.787572] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4172.790796] Lustre: Skipped 32 previous similar messages [ 4172.793153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4172.803696] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4172.807164] Lustre: Skipped 32 previous similar messages [ 4173.236133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4173.722342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.180157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.605196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4175.032048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4175.452184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4175.838158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4176.275582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4176.725321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.190089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.634638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4178.066987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4178.469339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4178.903032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4179.351600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4179.790646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.198506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.668031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.111947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.554219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.959137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.389854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.811732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4183.275472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4183.797508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4184.349814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4184.899827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.481874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4186.035925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4186.575836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4187.037189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4187.566548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4188.086430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4188.576760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4189.135851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4189.711670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.319400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.870075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.388116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.921534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4192.452297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4192.965238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4193.475151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4193.939331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4194.460915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4194.977478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4195.439683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4195.946261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4196.496064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4197.027432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4197.483348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4197.909615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4198.402809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4198.844182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4199.285387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4199.758502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4200.201776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4200.662549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4201.121771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4201.572197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4202.010748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4202.401816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4202.772535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.169190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.593036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.997560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.428217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.903524] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4204.909111] Lustre: Skipped 67 previous similar messages [ 4204.918456] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4204.922704] Lustre: Skipped 67 previous similar messages [ 4204.924437] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4204.928019] Lustre: Skipped 67 previous similar messages [ 4204.930643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.942575] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4204.948210] Lustre: Skipped 67 previous similar messages [ 4205.416724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4205.816238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4206.209335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4206.600817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4206.996236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4207.361813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4207.744452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4208.130806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4208.554774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4208.932482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4209.319846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4209.733764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4210.102977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4210.497357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4210.870944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4211.280643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4211.768403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4212.212929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4212.621651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4213.068903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4213.617950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4214.061153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4214.583662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4215.085362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4215.626777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4216.149071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4216.587519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.021918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.427317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.927597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4218.387088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4218.831819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4219.273697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4219.718533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4220.204531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4220.672806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4221.138288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4221.582288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.035088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.490390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.984498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4223.500689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4224.007074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4224.448697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4224.943993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4225.384208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4225.846806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4226.304707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4226.771670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4227.268044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4227.685283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4228.091575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4228.541871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4228.991390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4229.470699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4229.990403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4230.430234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4230.849983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4231.310558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4231.757922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4232.222985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4232.707832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4233.203240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4233.669392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4234.126812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4234.555576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4234.964852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4235.380361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4235.812279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4236.224858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4236.675796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4237.143629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4237.559613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4237.964294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4238.418340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4238.827900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4239.230597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4239.677668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4240.084616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4240.527104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4241.039895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4241.525111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4242.000034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4242.506830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4242.969898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4243.405483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4243.867391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4244.303244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4244.757562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4245.219351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4245.705710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4246.166456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4246.626286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4247.069466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4247.499706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4247.915677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4248.322469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4248.805560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4249.245741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4249.692117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4250.260792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4250.747967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4251.232274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4251.707677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4252.260753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4252.701241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4253.193915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4253.655261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4254.119794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4254.554429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4255.010114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4255.458567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4255.914534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4256.358735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4256.845444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4257.291816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4257.697390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4258.107808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4258.528798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4258.985202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4259.457822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4259.943358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4260.395966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4260.896926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4261.370952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4261.808979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4262.252932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4262.685535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4263.102969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4263.508882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4263.923827] LustreError: 162794:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4263.928419] LustreError: 162794:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4263.933214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4264.328722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4264.711621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4265.136790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4265.522143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4265.909838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4266.284438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4266.670399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4267.058889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4267.445862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4267.846504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4268.234226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4268.702062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4269.096892] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4269.102017] Lustre: Skipped 143 previous similar messages [ 4269.110603] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4269.116684] Lustre: Skipped 143 previous similar messages [ 4269.120253] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4269.122398] Lustre: Skipped 143 previous similar messages [ 4269.124940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4269.130605] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4269.133607] Lustre: Skipped 143 previous similar messages [ 4269.570259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4270.031132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4270.481522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4270.849860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4271.264033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4271.729811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4272.210389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4272.677878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4273.101738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4273.498898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4273.918821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4274.326823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4274.758439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4275.238273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4275.674372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4276.093946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4276.542759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4276.962371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4277.378953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4277.779531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4278.193859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4278.600492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4278.990398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4279.378026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4279.794717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4280.234068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4280.737542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4281.202095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4281.722933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4282.274163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4282.756875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4283.184726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4283.614759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4284.030277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4284.429972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4284.888851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4285.332081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4285.753040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4286.188721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4286.614210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4287.009718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4287.405672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4287.802623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4288.193728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4288.576484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4288.957847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4289.360159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4289.764944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4290.183404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4290.627409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4291.092940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4291.520512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4291.991805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4292.419301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4292.855568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4293.343683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4293.835361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4294.290858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4294.768411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4295.231624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4295.653927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4296.152732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4296.581671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4297.055589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4297.524078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4297.953105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4298.388723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4298.817200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4299.242115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4299.655607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4300.062974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4300.496966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4300.898947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4301.291803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4301.694209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4302.097813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4302.521905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4302.950493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4303.368894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4303.774062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4304.176132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4304.586733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4304.998221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4305.411035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4305.812317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4306.224113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4306.629774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4307.066375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4307.535188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4307.987486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4308.426413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4308.888424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4309.367886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4309.806576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4310.271420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4310.748684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4311.207446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4311.645660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4312.129304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4312.581235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4313.013142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4313.459619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4313.900699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4314.338422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4314.779374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4315.288958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4315.741382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4316.209962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4316.651833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4317.095829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4317.546117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4317.985162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4318.434680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4318.879448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4319.336378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4319.763636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4320.217917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4320.659887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4321.067570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4321.525575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4321.961495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4322.386830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4322.811415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4323.233335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4323.654433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4324.092355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4324.553463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4325.020163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4325.507674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4325.962934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4326.415782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4326.953442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4327.438907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4327.925453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4328.423198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4328.919641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4329.386795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4329.854415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4330.296047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4330.704959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4331.133090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4331.550236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4332.028270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4332.529379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4333.024865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4333.462465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4333.973879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4334.472118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4334.883574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4335.336636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4335.801488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4336.255141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4336.710445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4337.202544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4337.735429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4338.162948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4338.634362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4339.064654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4339.505681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4339.923267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4340.375542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4340.865477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4341.307546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4341.778990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4342.294197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4342.764620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4343.245959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4343.706658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4344.152371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4344.567450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4344.991523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4345.417289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4345.846240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4346.291164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4346.765840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4347.328853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4347.786755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4348.253919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4348.723968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4349.160442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4349.570911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4349.990647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4350.409555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4350.883034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4351.360445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4351.827485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4352.365944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4352.831698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4353.261496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4353.709608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4354.141669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4354.555191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4355.019703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4355.474490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4355.913770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4356.365410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4356.798958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4357.255425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4357.664257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4358.078324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4358.482195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4358.913959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4359.334172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4359.768602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4360.266518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4360.725087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4361.169948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4361.622050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4362.030822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4362.432798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4362.886655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4363.333576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4363.846160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4364.387859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4364.794313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4365.246724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4365.719520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4366.142535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4366.586420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4367.044039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4367.531456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4367.985671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4368.439852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4368.855936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4369.290843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4369.742719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4370.175923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4370.703080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4371.237157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4371.764296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4372.212205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4372.737261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4373.216470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4373.652595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4374.117189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4374.581978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4375.027382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4375.460188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4375.942182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4376.433699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4376.935882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4377.423903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4377.982379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4378.536370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4379.140096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4379.662837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4380.189655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4380.708782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4381.218570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4381.738983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4382.238155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4382.729655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4383.182666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4383.630648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4384.058621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4384.478930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4384.934440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4385.341684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4385.741763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4386.168414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4386.579328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4387.014027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4387.424438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4387.831594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4388.236341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4388.663646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4389.100903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4389.514132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4389.953375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4390.409380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4390.822977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4391.266989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4391.696365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4392.081712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4392.491423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4392.892542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4393.305579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4393.714219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4394.183791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4394.618631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4395.035678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4395.447492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4395.910084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4396.341858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4396.750690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4397.148485] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4397.151421] Lustre: Skipped 285 previous similar messages [ 4397.158397] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4397.163850] Lustre: Skipped 285 previous similar messages [ 4397.166424] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4397.169715] Lustre: Skipped 285 previous similar messages [ 4397.172025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4397.179334] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4397.181993] Lustre: Skipped 285 previous similar messages [ 4397.616150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4398.050505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4398.460466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4398.951943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4399.458639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4399.955746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4400.444589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4400.996794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4401.536967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4402.089660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4402.645339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4403.256833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4403.781152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4404.342620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4404.932562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4405.516140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4406.127489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4406.697158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4407.260824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4407.801451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4408.296836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4408.803116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4409.322187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4409.832239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4410.412634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4410.919950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4411.409541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4411.941600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4412.457867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4412.922324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4413.462319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4413.932117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4414.449675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4414.957514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4415.503753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4416.017726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4416.528489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4417.076518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4417.552143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4417.990681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4418.506828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4418.982060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4419.426035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4419.893484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4420.373172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4420.889585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4421.354652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4421.800399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4422.217110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4422.646089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4423.062109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4423.462580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4423.885186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4424.291046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4424.722951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4425.183176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4425.658882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4426.080177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4426.511905] LustreError: 162794:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4426.517639] LustreError: 162794:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4426.518162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4426.931867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4427.350227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4427.806964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4428.243000] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4428.688612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4429.146105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4429.649289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4430.128250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4430.613458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4431.083498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4431.581461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4432.071445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4432.560605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4433.040544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4433.488413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4433.941522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4434.462797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4435.017833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4435.550492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.106674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.633981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4437.160400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4437.648470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4438.179107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4438.692496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4439.211102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4439.717179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4440.209876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4440.775092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4441.307335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4441.822498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4442.364180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4442.836733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4443.459657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4444.091107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4444.613146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4445.151343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4445.678573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4446.258283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4446.739976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4447.250669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4447.793160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4448.397894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4448.882659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4449.462749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4450.029673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4450.576115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4451.046918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4451.609277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4452.131801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4452.691914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4453.225477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4453.757462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4454.285501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4454.705694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4455.205886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4455.668658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4456.098389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4456.590876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4457.076093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4457.554821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4458.069761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4458.579499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4459.182238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4459.690900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4460.275533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4460.807960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4461.343906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4461.889723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4462.471731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4462.989593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4463.559439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4464.126263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4464.664925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4465.209053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4465.743310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4466.319466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4466.840914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4467.383155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4467.846757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4468.372196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4468.837749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4469.335294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4469.823874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4470.308470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4470.791809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4471.293688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4471.807722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4472.328466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4472.804780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4473.285630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4473.734407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4474.168935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4474.599983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4475.085783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4475.547472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4476.019664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4476.472927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4476.913711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4477.339234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4477.765032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4478.208455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4478.644259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4479.110091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4479.531252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4479.977362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4480.389843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4480.812386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4481.221145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4481.607583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4482.009673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4482.430918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4482.825399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4483.224528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4483.639883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4484.063092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4484.518814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4484.979750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4485.426917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4485.870579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4486.328850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4486.846158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4487.309784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4487.766427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4488.235942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4488.681820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4489.130223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4489.595898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4490.050087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4490.530700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4490.970390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4491.406416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4491.858623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4492.285857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4492.698623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4493.143903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4493.541414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4493.955307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4494.390096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4494.780617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4495.184376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4495.670055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4496.092529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4496.528038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4497.005098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4497.494355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4497.925977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4498.351266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4498.789438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4499.231537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4499.687635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4500.127657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4500.571945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4501.022173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4501.455835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4501.937227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4502.388144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4502.826823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4503.246802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4503.677980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4504.110647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4504.598276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4505.087132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4505.522622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4505.980548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4506.413956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4506.851602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4507.305934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4507.742850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4508.193853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4508.647570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4509.101317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4509.559429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4510.007060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4510.479180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4511.019038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4511.529079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4512.044467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4512.508993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4512.966823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4513.413256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4513.925878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4514.439411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4514.941838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4515.426610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4515.889155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4516.356593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4516.835878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4517.319210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4517.765797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4518.238290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4518.693270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4519.130164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4519.577797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4520.033545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4520.492175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4520.948580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4521.430986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4521.877187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4522.334157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4522.814180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4523.261753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4523.686429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4524.101542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4524.600217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4525.071825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4525.533658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4525.991885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4526.422494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4526.875528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4527.312942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4527.798963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4528.253946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4528.704194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4529.153385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4529.578162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4530.018041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4530.482839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4530.925049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4531.358990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4531.761598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4532.209667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4532.708024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4533.122565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4533.546192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4534.000142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4534.448272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4534.924428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4535.467249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4536.007936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4536.504637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4537.113486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4537.817841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4538.366655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4538.904592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4539.481461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4539.997665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4540.561341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4541.087926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4541.673389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4542.231603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4542.787649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4543.350541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4543.822927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4544.316629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4544.830633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4545.343875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4545.829038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4546.317684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4546.794368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4547.282062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4547.823341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4548.307421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4548.777145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4549.264426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4549.695783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4550.188348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4550.647605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4551.094357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4551.551134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4551.969200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4552.456077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4552.866533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4553.309917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4553.733242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4554.158797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4554.595880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4555.015369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4555.492672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4555.969963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4556.437369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4556.918942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4557.362034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4557.835690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4558.292955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4558.829930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4559.267934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4559.705443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4560.158902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4560.601043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4561.062442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4561.590643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4562.090233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4562.650412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4563.170709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4563.671297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4564.147850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4564.587605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4565.045455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4565.503758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4566.021687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4566.506291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4566.955159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4567.477114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4567.946979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4568.440207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4568.959184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4569.430643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4569.957506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4570.482281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4571.030745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4571.622395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4572.230465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4572.789450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4573.319412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4573.865691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4574.346313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4574.832879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4575.426449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4575.939662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4576.525488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4577.031632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4577.531388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4578.033829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4578.527956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4579.015882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4579.457033] LustreError: 162794:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4579.463547] LustreError: 162794:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4579.464624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4579.932486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4580.358972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4580.802584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4581.207426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4581.625478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4582.057525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4582.491382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4582.899531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4583.351858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4583.820521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4584.255736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4584.703090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4585.169287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4585.609050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4586.042651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4586.507563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4586.953261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4587.445909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4587.970644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4588.439862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4588.921398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4589.422383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4589.884156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4590.349945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4590.835404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4591.281808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4591.763920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4592.211979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4592.651504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4593.127398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4593.569475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4594.031224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4594.483406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4594.915697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4595.365684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4595.783132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4596.209750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4596.661150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4597.073819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4597.494924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4597.936934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4598.426288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4598.927291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4599.433755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4599.972410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4600.450435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4600.893696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4601.355614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4601.837759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4602.325143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4602.858996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4603.329067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4603.806730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4604.339986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.594664] LustreError: 164370:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 4835.599276] LustreError: 162794:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 5005.283547] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5005.285553] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5005.289484] Lustre: Skipped 431 previous similar messages [ 5010.302300] LustreError: 216597:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5010.389409] Lustre: server umount lustre-OST0000 complete [ 5012.959972] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5012.964132] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5012.966497] Lustre: Skipped 1 previous similar message [ 5015.520590] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5015.522570] Lustre: Skipped 1 previous similar message [ 5020.640466] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5020.643568] Lustre: Skipped 1 previous similar message [ 5024.737655] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5024.740566] Lustre: Skipped 1 previous similar message [ 5026.271179] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5026.314366] LustreError: 216799:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 5026.317535] LustreError: 216799:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5026.424097] Lustre: server umount lustre-MDT0000 complete [ 5027.941273] LustreError: 167188:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763158319 with bad export cookie 4215268244791494821 [ 5027.943865] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5027.945505] LustreError: 167188:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5028.019853] LustreError: 217002:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5028.022918] LustreError: 217002:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5028.098941] Lustre: server umount lustre-MDT0001 complete [ 5032.339153] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 5033.626176] Key type lgssc unregistered [ 5033.782558] LNet: 217534:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5033.785737] LNetError: 217534:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5033.797417] LNet: Removed LNI 192.168.204.143@tcp [ 5034.154101] Key type .llcrypt unregistered [ 5034.155693] Key type ._llcrypt unregistered [ 5044.943241] Key type ._llcrypt registered [ 5044.944618] Key type .llcrypt registered [ 5045.001175] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 5045.379802] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5045.419264] alg: No test for adler32 (adler32-zlib) [ 5046.322211] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 5046.432596] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 5048.031906] Key type lgssc registered [ 5048.527365] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5051.689496] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5051.697461] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5052.922452] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5054.529599] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5057.911392] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5058.060870] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5059.656247] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5061.897354] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5063.354443] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5066.040053] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5066.186822] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5068.458398] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5070.513654] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5079.530988] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 5083.236329] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5083.853635] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5083.860036] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5083.864106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.868299] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5083.868312] LustreError: 218927:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 5084.644449] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5084.645303] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5084.647986] Lustre: Skipped 1 previous similar message [ 5084.652557] Lustre: Skipped 1 previous similar message [ 5086.422828] Lustre: lustre-OST0000: Not available for connect from 192.168.204.43@tcp (stopping) [ 5086.575195] LustreError: 221324:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5086.623399] Lustre: server umount lustre-OST0000 complete [ 5103.943128] LustreError: 218927:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 5114.905380] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5115.030532] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5117.116776] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5119.121288] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5125.285787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.289730] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5125.289754] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.849691] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 7 sec [ 5128.097179] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9cc1c3583000.ost_server_uuid 50 [ 5128.650472] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9cc1c3583000.ost_server_uuid in FULL state after 0 sec [ 5130.724193] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5130.730683] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5130.730819] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5130.736468] Lustre: Skipped 1 previous similar message [ 5135.724086] LustreError: 222643:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5135.726436] LustreError: 222643:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5135.765947] Lustre: server umount lustre-OST0000 complete [ 5139.936145] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5139.938903] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5139.943060] Lustre: Skipped 1 previous similar message [ 5139.945150] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5143.164982] LustreError: 222845:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 5143.167364] LustreError: 222845:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5143.196356] Lustre: server umount lustre-MDT0000 complete [ 5144.597471] LustreError: 219624:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763158436 with bad export cookie 3053213144347936151 [ 5144.600556] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5144.601454] LustreError: 219624:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5144.732612] Lustre: server umount lustre-MDT0001 complete [ 5148.770877] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 5149.878523] Key type lgssc unregistered [ 5150.013415] LNet: 223577:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5150.016263] LNetError: 223577:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5150.025381] LNet: Removed LNI 192.168.204.143@tcp [ 5150.359150] Key type .llcrypt unregistered [ 5150.360186] Key type ._llcrypt unregistered [ 5162.133450] Key type ._llcrypt registered [ 5162.134618] Key type .llcrypt registered [ 5162.171866] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 5162.562553] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5162.574158] alg: No test for adler32 (adler32-zlib) [ 5163.419341] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 5163.509127] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 5165.087213] Key type lgssc registered [ 5165.459771] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5168.616122] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5168.624173] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5169.794614] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5171.128814] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5174.074452] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5174.197602] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5175.558919] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5176.519950] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5178.785891] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5178.907457] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5179.943438] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 5180.848286] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5183.702181] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5183.721470] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5183.786260] Lustre: lustre-OST0001: new disk, initializing [ 5183.788301] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5183.824713] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5184.706046] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5184.709244] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5184.761982] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5185.679351] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5191.150822] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5198.813634] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5215.199889] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5215.202424] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5215.207894] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5216.737531] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5216.743100] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5216.746550] Lustre: Skipped 2 previous similar messages [ 5218.432716] LustreError: 228625:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 5218.470536] Lustre: server umount lustre-MDT0000 complete [ 5219.774555] LustreError: 225246:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763158511 with bad export cookie 5809463704201179606 [ 5219.777395] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5219.779057] LustreError: 225246:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5219.858375] LustreError: 228827:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 5219.860508] LustreError: 228827:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5219.923183] Lustre: server umount lustre-MDT0001 complete [ 5231.028100] LustreError: 229027:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5231.031103] LustreError: 229027:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5231.049214] Lustre: server umount lustre-OST0000 complete [ 5242.611137] LustreError: 229229:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5242.614100] LustreError: 229229:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5242.661934] Lustre: server umount lustre-OST0001 complete [ 5243.713194] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5243.750135] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5244.130167] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5244.163569] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5244.567595] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5244.609079] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5245.005269] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5245.045404] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5249.794617] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 5253.361730] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5253.504948] LustreError: 230636:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5253.513755] LustreError: 230636:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5253.527034] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5254.792172] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5257.745142] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5259.178901] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5260.193299] Lustre: 231748:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5262.404892] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5262.515329] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5262.517876] Lustre: Skipped 1 previous similar message [ 5263.524686] LustreError: 232103:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5263.529782] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49761) [ 5264.394560] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5267.440080] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5268.590903] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 5269.578228] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5272.837797] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5274.137575] Lustre: 233592:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5283.808169] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5283.810896] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5283.816349] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5283.819061] Lustre: Skipped 3 previous similar messages [ 5289.442043] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5289.444510] Lustre: Skipped 7 previous similar messages [ 5289.855226] LustreError: 233900:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5289.857731] LustreError: 233900:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5289.891251] Lustre: server umount mylustre-MDT0000 complete [ 5291.251278] LustreError: 231365:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763158582 with bad export cookie 5809463704201182427 [ 5291.253461] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5291.256194] LustreError: 231365:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5291.390275] Lustre: server umount mylustre-MDT0001 complete [ 5302.772067] LustreError: 234301:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5302.774219] LustreError: 234301:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 5302.790703] Lustre: server umount mylustre-OST0000 complete [ 5314.396266] Lustre: server umount mylustre-OST0001 complete [ 5315.493818] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5315.522686] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5315.916655] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5315.953574] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5316.324094] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5316.355933] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5316.722782] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5316.748191] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5321.675514] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 5325.111248] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5325.261404] LustreError: 235913:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5325.286184] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5325.288470] Lustre: Skipped 1 previous similar message [ 5326.611390] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5329.516447] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5330.967492] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5331.932363] Lustre: 237024:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5334.235716] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5335.397366] LustreError: 237382:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5335.402067] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49793) [ 5336.232345] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5339.194233] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5340.329822] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 5341.177139] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5344.385438] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5345.600700] Lustre: 238870:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5355.488151] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5355.490258] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5355.494625] Lustre: Skipped 3 previous similar messages [ 5355.496862] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 5358.719408] LustreError: 239120:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5358.721455] LustreError: 239120:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 5358.759150] Lustre: server umount tfs-MDT0000 complete [ 5359.010245] LustreError: 235912:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5359.016597] LustreError: 235912:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5360.147439] LustreError: 235888:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763158651 with bad export cookie 5809463704201184380 [ 5360.151448] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5360.152222] LustreError: 235888:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5360.279217] Lustre: server umount tfs-MDT0001 complete [ 5372.423803] Lustre: server umount tfs-OST0000 complete [ 5384.036514] Lustre: server umount tfs-OST0001 complete [ 5385.124483] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5385.155307] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5385.551095] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5385.587132] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5385.957382] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5385.987115] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5386.361889] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5386.392158] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5391.362693] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 5395.316648] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5395.513214] LustreError: 241132:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5395.539570] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5395.542392] Lustre: Skipped 3 previous similar messages [ 5397.132965] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5400.338542] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5401.902783] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5402.978770] Lustre: 242242:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5405.548283] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5407.223854] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49825) [ 5407.766721] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5410.286801] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 5410.775275] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5412.893253] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5416.116972] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5417.418705] Lustre: 244086:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5424.767577] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 17:18:35 (1763158715) [ 5441.508820] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5441.511854] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5441.514199] Lustre: Skipped 6 previous similar messages [ 5441.517178] Lustre: Skipped 5 previous similar messages [ 5441.518766] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5442.750761] LustreError: 244457:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5442.752555] LustreError: 244457:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 5442.781397] Lustre: server umount lustre-MDT0000 complete [ 5444.193434] LustreError: 244090:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763158735 with bad export cookie 5809463704201186074 [ 5444.198874] LustreError: 244090:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5444.200038] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5446.624480] LustreError: 241126:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5446.631241] LustreError: 241126:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 5450.351372] Lustre: server umount lustre-MDT0001 complete [ 5457.721701] Lustre: server umount lustre-OST0000 complete [ 5473.247159] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5473.309910] Lustre: server umount lustre-OST0001 complete [ 5475.548510] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 5479.032722] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 5482.661629] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5484.912839] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5487.003764] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5489.228050] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5491.650691] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5491.670341] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 5491.670341] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 5491.670341] [ 5491.676979] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 5491.765132] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5491.778844] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5491.815163] Lustre: lustre-MDT0000: new disk, initializing [ 5491.843273] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5491.845500] Lustre: Skipped 3 previous similar messages [ 5491.852314] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5493.229257] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5497.603394] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5497.627165] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5497.661797] Lustre: 247247:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5497.683344] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5497.685710] Lustre: Skipped 1 previous similar message [ 5497.721273] Lustre: lustre-MDT0001: new disk, initializing [ 5497.763024] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5497.767711] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5498.995090] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5503.041792] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5503.070791] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5503.176286] Lustre: lustre-OST0000: new disk, initializing [ 5503.178588] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5504.500571] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5504.503339] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5504.529972] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5505.018885] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5509.089284] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5509.110378] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5509.159061] Lustre: lustre-OST0001: new disk, initializing [ 5509.161251] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5511.030209] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5511.046717] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5511.050846] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5511.099685] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5515.142458] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 17:20:05 (1763158805) [ 5518.346852] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 17:20:08 (1763158808) [ 5530.080159] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5530.080533] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5530.082691] LustreError: Skipped 1 previous similar message [ 5530.083231] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5530.087204] Lustre: Skipped 3 previous similar messages [ 5530.091645] Lustre: Skipped 5 previous similar messages [ 5535.789943] LustreError: 250668:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5535.792012] LustreError: 250668:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 5535.811207] Lustre: server umount lustre-OST0000 complete [ 5536.740270] LustreError: 248784:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5536.746492] LustreError: 248784:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5538.784201] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5543.447971] Lustre: server umount lustre-MDT0000 complete [ 5544.663233] LustreError: 249654:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763158836 with bad export cookie 5809463704201187628 [ 5544.666985] LustreError: 249654:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5544.667526] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5546.976554] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5546.981597] Lustre: Skipped 3 previous similar messages [ 5550.830750] Lustre: server umount lustre-MDT0001 complete [ 5554.090070] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 5563.732402] Lustre: server umount lustre-OST0001 complete [ 5565.664714] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 5571.520621] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 5575.008131] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5577.180312] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5579.218049] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5581.384321] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5583.738035] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5583.759453] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5583.840256] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5583.854230] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5583.891618] Lustre: lustre-MDT0000: new disk, initializing [ 5583.920933] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5585.179474] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5589.358949] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5589.380940] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5589.409453] Lustre: 254125:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5589.511968] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5590.845681] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5593.979559] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5595.235553] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5597.619121] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5597.645385] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5599.301839] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5599.361731] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5599.895210] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5603.078394] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5609.954084] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5609.954241] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5609.957018] LustreError: Skipped 1 previous similar message [ 5609.961395] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5609.964537] Lustre: Skipped 9 previous similar messages [ 5611.598979] Lustre: server umount lustre-OST0000 complete [ 5627.359142] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5627.431268] Lustre: server umount lustre-MDT0000 complete [ 5628.664255] LustreError: 254906:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763158920 with bad export cookie 5809463704201189098 [ 5628.667233] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5628.667849] LustreError: 254906:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5628.785475] Lustre: server umount lustre-MDT0001 complete [ 5632.704881] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 5633.804281] Key type lgssc unregistered [ 5633.952414] LNet: 257502:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5633.954773] LNetError: 257502:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5633.963372] LNet: Removed LNI 192.168.204.143@tcp [ 5634.320145] Key type .llcrypt unregistered [ 5634.321454] Key type ._llcrypt unregistered [ 5644.603301] Key type ._llcrypt registered [ 5644.604739] Key type .llcrypt registered [ 5644.647475] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 5645.020623] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5645.059741] alg: No test for adler32 (adler32-zlib) [ 5645.950638] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 5646.048124] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 5647.639220] Key type lgssc registered [ 5647.990940] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5650.457694] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5650.464038] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5651.605543] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5652.795054] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5655.598087] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5655.735369] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5657.141271] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5658.968826] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5660.190488] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5662.569797] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5662.716027] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5664.713840] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5666.630403] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5667.001155] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 5670.882445] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5670.886979] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5670.887356] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5673.773406] LustreError: 261215:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5673.815914] Lustre: server umount lustre-OST0000 complete [ 5676.000714] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5676.001370] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5676.005622] Lustre: Skipped 1 previous similar message [ 5681.120238] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5681.122328] Lustre: Skipped 2 previous similar messages [ 5681.343906] LustreError: 261421:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 5681.345753] LustreError: 261421:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5681.376481] Lustre: server umount lustre-MDT0000 complete [ 5682.686500] LustreError: 258902:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763158974 with bad export cookie 1843071722479981989 [ 5682.689643] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5682.690310] LustreError: 258902:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5682.763023] LustreError: 261623:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5682.765514] LustreError: 261623:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5682.818894] Lustre: server umount lustre-MDT0001 complete [ 5686.738477] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 5687.845961] Key type lgssc unregistered [ 5687.980440] LNet: 262153:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5687.983839] LNetError: 262153:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5687.995277] LNet: Removed LNI 192.168.204.143@tcp [ 5688.305080] Key type .llcrypt unregistered [ 5688.306769] Key type ._llcrypt unregistered [ 5695.109457] Key type ._llcrypt registered [ 5695.110537] Key type .llcrypt registered [ 5695.164973] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 5695.606040] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5695.671100] alg: No test for adler32 (adler32-zlib) [ 5696.546062] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 5696.655375] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 5698.247202] Key type lgssc registered [ 5698.629299] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5701.762856] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5701.771971] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5717.215390] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 5722.335219] LustreError: 263303:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.143@tcp: failed processing log, type 4: rc = -110 [ 5748.959239] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5751.009725] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5752.928074] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5753.742672] LustreError: 263967:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 5753.788089] Lustre: server umount lustre-OST0000 complete [ 5759.316173] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 5760.384511] Key type lgssc unregistered [ 5760.513585] LNet: 264802:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5760.517203] LNetError: 264802:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5760.526535] LNet: Removed LNI 192.168.204.143@tcp [ 5760.866166] Key type .llcrypt unregistered [ 5760.867403] Key type ._llcrypt unregistered [ 5775.182787] Key type ._llcrypt registered [ 5775.184238] Key type .llcrypt registered [ 5775.222976] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 5775.633234] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5775.659833] alg: No test for adler32 (adler32-zlib) [ 5776.530727] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 5776.631121] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 5778.223224] Key type lgssc registered [ 5778.632608] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5783.309024] loop0: detected capacity change from 0 to 204800000 [ 5783.384476] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5783.389277] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5783.526084] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5783.547496] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5783.923106] loop0: detected capacity change from 0 to 204800000 [ 5783.992287] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5783.996078] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5784.133760] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5784.152406] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5784.537176] loop0: detected capacity change from 0 to 204800000 [ 5784.617264] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5784.621771] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5784.668441] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5784.723776] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5785.094751] loop0: detected capacity change from 0 to 204800000 [ 5785.150906] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5785.239801] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5785.588687] loop0: detected capacity change from 0 to 204800000 [ 5785.598913] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5787.015408] loop0: detected capacity change from 0 to 204800000 [ 5787.031529] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5788.396695] loop0: detected capacity change from 0 to 204800000 [ 5788.409581] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5789.841345] loop0: detected capacity change from 0 to 204800000 [ 5789.850793] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5791.252346] loop0: detected capacity change from 0 to 204800000 [ 5791.294544] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5791.322846] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5791.330036] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5791.354316] Lustre: 268176:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5791.358124] Lustre: 268176:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 5791.362086] Lustre: 268176:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 5791.366463] Lustre: 268176:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 5791.370701] Lustre: 268176:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 5791.374992] Lustre: 268176:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 5791.379332] Lustre: 268176:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 5791.383138] Lustre: 268176:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 5791.388733] Lustre: 268176:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 5791.393328] Lustre: 268176:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 5791.399239] Lustre: 268176:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 5791.403653] Lustre: 268176:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 5791.409155] Lustre: 268176:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 5791.412562] Lustre: 268176:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 5794.147107] LustreError: 268442:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5794.200452] Lustre: server umount MGS complete [ 5794.547183] loop0: detected capacity change from 0 to 204800000 [ 5794.588571] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5794.782874] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5794.803387] Lustre: lustre-MDT0000: Aborting client recovery [ 5795.166358] loop1: detected capacity change from 0 to 204800000 [ 5795.231847] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5795.266608] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5795.293560] Lustre: 268611:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5795.297043] Lustre: 268611:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 5795.301672] Lustre: 268611:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 5795.306085] Lustre: 268611:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 5795.310923] Lustre: 268611:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 5795.316298] Lustre: 268611:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 5795.321815] Lustre: 268611:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 5795.331990] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5795.423761] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5795.449517] Lustre: lustre-MDT0001: Aborting client recovery [ 5795.810114] loop2: detected capacity change from 0 to 204800000 [ 5795.866953] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5795.912413] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5795.936049] Lustre: 268695:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5795.940717] Lustre: 268695:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 5795.946322] Lustre: 268695:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 5795.952337] Lustre: 268695:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 5795.958868] Lustre: 268695:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 5795.965994] Lustre: 268695:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 5795.972860] Lustre: 268695:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 5796.047176] Lustre: Found index 0 for lustre-OST0000, updating log [ 5796.116280] Lustre: lustre-OST0000: Aborting recovery [ 5796.487171] loop3: detected capacity change from 0 to 204800000 [ 5796.538393] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5796.580108] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5796.609433] Lustre: 268783:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5796.615717] Lustre: 268783:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 5796.622180] Lustre: 268783:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 5796.626790] Lustre: 268783:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 5796.630175] Lustre: 268783:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 5796.633850] Lustre: 268783:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 5796.638261] Lustre: 268783:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 5796.685246] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5796.687199] Lustre: Skipped 1 previous similar message [ 5796.695642] Lustre: lustre-OST0001: Aborting recovery [ 5801.960991] LustreError: 268554:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 5805.026571] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 5805.027712] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 5805.028408] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 5805.032977] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 5805.038927] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 5805.038944] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 5807.292197] LustreError: 268521:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 258us: rc = -2 [ 5820.385877] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5820.386539] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5820.389419] Lustre: Skipped 3 previous similar messages [ 5820.391249] Lustre: Skipped 2 previous similar messages [ 5824.700991] LustreError: 269137:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5824.703050] LustreError: 269137:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 5824.741413] Lustre: server umount lustre-MDT0000 complete [ 5825.078052] LustreError: 268504:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763159116 with bad export cookie 15159540845159458466 [ 5825.080569] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5825.082553] LustreError: 268504:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5825.504034] LustreError: 269018:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5825.504269] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5825.504765] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5825.504772] Lustre: Skipped 1 previous similar message [ 5825.510541] LustreError: 269018:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5825.516948] Lustre: Skipped 1 previous similar message [ 5830.623491] LustreError: 269018:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5830.623732] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5830.628029] LustreError: 269018:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5830.629893] Lustre: Skipped 2 previous similar messages [ 5831.235037] LustreError: 269191:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 5831.238286] LustreError: 269191:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5831.306831] Lustre: server umount lustre-MDT0001 complete [ 5837.740172] LustreError: 269242:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5837.742475] LustreError: 269242:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5837.771074] Lustre: server umount lustre-OST0000 complete [ 5844.202584] LustreError: 269293:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5844.205428] LustreError: 269293:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5844.258175] Lustre: server umount lustre-OST0001 complete [ 5846.956155] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 17:25:37 (1763159137) [ 5856.634228] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 5859.955220] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 5863.559292] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5865.754295] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5867.855387] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5870.026093] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5872.495699] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5872.522383] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5872.624718] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5872.637450] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5872.674977] Lustre: lustre-MDT0000: new disk, initializing [ 5872.700756] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5872.707639] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5874.092835] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5878.447201] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5878.472317] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5878.506385] Lustre: 272836:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5878.534305] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5878.536395] Lustre: Skipped 1 previous similar message [ 5878.575910] Lustre: lustre-MDT0001: new disk, initializing [ 5878.610898] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5878.626900] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5878.630839] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5879.983029] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5883.142394] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5884.429165] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5886.845093] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5886.872511] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5886.978463] Lustre: lustre-OST0000: new disk, initializing [ 5886.981297] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5887.016717] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5889.035397] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5890.939508] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5890.944099] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5890.994916] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5892.288991] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5894.724185] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 5898.542071] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 5908.387559] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 5947.361202] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5947.361631] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5947.364424] LustreError: Skipped 1 previous similar message [ 5947.369253] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5947.372875] Lustre: Skipped 1 previous similar message [ 5952.482080] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5952.484952] Lustre: Skipped 1 previous similar message [ 5958.111112] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5958.133124] LustreError: 276492:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5958.135576] LustreError: 276492:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5958.180551] Lustre: server umount lustre-OST0000 complete [ 5960.671954] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5960.675945] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5960.682372] Lustre: Skipped 1 previous similar message [ 5960.685252] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5960.687635] Lustre: Skipped 2 previous similar messages [ 5965.669288] Lustre: server umount lustre-MDT0000 complete [ 5966.945482] LustreError: 272827:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763159258 with bad export cookie 15159540845159461175 [ 5966.949848] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5967.064731] Lustre: server umount lustre-MDT0001 complete [ 5970.090251] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5971.502501] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5973.250064] Lustre: server umount MGS complete [ 5977.261482] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5977.438112] LustreError: 278462:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5977.445620] LustreError: 278474:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 5977.470870] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5978.884355] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5981.969896] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5983.573962] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5985.597714] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5986.948783] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5989.252481] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5991.247600] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 5993.058098] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6028.256507] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6028.259238] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6028.264079] Lustre: Skipped 1 previous similar message [ 6028.265804] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6028.268127] Lustre: Skipped 2 previous similar messages [ 6032.354344] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6033.325117] LustreError: 281141:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6033.326756] LustreError: 281141:0:(obd_class.h:479:obd_check_dev()) Skipped 19 previous similar messages [ 6033.365699] Lustre: server umount lustre-OST0000 complete [ 6037.472773] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6037.476125] Lustre: Skipped 1 previous similar message [ 6040.797835] Lustre: server umount lustre-MDT0000 complete [ 6042.028431] LustreError: 278443:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763159333 with bad export cookie 15159540845159462988 [ 6042.031549] LustreError: 278443:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 6042.032070] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6042.155254] Lustre: server umount lustre-MDT0001 complete [ 6045.921739] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 6046.972759] Key type lgssc unregistered [ 6047.111471] LNet: 282074:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6047.114295] LNetError: 282074:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6047.126457] LNet: Removed LNI 192.168.204.143@tcp [ 6047.435135] Key type .llcrypt unregistered [ 6047.436334] Key type ._llcrypt unregistered [ 6059.698769] Key type ._llcrypt registered [ 6059.700160] Key type .llcrypt registered [ 6059.739776] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 6065.837558] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 6066.188059] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6066.194496] alg: No test for adler32 (adler32-zlib) [ 6067.069055] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 6067.162623] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 6068.743155] Key type lgssc registered [ 6069.167920] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6072.265113] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6074.393735] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6076.467222] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6078.714991] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6081.083536] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6081.104847] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6081.110675] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6082.209374] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6082.222844] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6082.261049] Lustre: lustre-MDT0000: new disk, initializing [ 6082.287337] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6082.294400] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6083.634937] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6087.797636] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6087.825459] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6087.866715] Lustre: 285990:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6087.899852] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6087.901649] Lustre: Skipped 1 previous similar message [ 6087.943979] Lustre: lustre-MDT0001: new disk, initializing [ 6087.983038] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6087.997281] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6088.001317] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6089.440145] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6092.654263] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6093.907525] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6096.346355] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6096.370982] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6096.485850] Lustre: lustre-OST0000: new disk, initializing [ 6096.488612] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6096.523212] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6098.542852] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6098.876089] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6098.880742] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6098.924179] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6101.903392] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6105.338403] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 6107.883943] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 6116.365090] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 6160.353640] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6160.356063] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6160.356465] LustreError: Skipped 1 previous similar message [ 6160.360416] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6160.363141] Lustre: Skipped 1 previous similar message [ 6162.991086] LustreError: 289551:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 6163.029846] Lustre: server umount lustre-OST0000 complete [ 6164.960028] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6164.963077] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6164.967268] Lustre: Skipped 1 previous similar message [ 6164.968993] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6170.431444] LustreError: 289752:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 6170.434205] LustreError: 289752:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6170.464117] Lustre: server umount lustre-MDT0000 complete [ 6171.743462] LustreError: 286769:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763159463 with bad export cookie 12960660222357077320 [ 6171.747205] LustreError: 286769:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6171.748699] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6171.817962] LustreError: 289953:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6171.819757] LustreError: 289953:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6171.878740] Lustre: server umount lustre-MDT0001 complete [ 6174.718724] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6176.135919] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6177.837615] LustreError: 290894:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6177.839666] LustreError: 290894:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6177.864984] Lustre: server umount MGS complete [ 6182.052806] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6182.222419] LustreError: 291516:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6182.227245] LustreError: 291516:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6182.234102] LustreError: 291528:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 6182.267297] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6183.648250] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6186.699350] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6186.826781] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6188.294666] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6190.222678] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6191.427467] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6193.974580] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6196.292633] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6198.299315] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6233.059794] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6233.060395] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6233.061290] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6233.061296] Lustre: Skipped 2 previous similar messages [ 6233.062779] LustreError: Skipped 1 previous similar message [ 6233.066436] Lustre: Skipped 2 previous similar messages [ 6237.805192] LustreError: 294197:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6237.807760] LustreError: 294197:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 6237.852832] Lustre: server umount lustre-OST0000 complete [ 6243.296583] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6243.296864] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6243.297817] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6243.297823] Lustre: Skipped 1 previous similar message [ 6243.305985] Lustre: Skipped 1 previous similar message [ 6245.398398] Lustre: server umount lustre-MDT0000 complete [ 6246.669521] LustreError: 292235:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763159538 with bad export cookie 12960660222357079140 [ 6246.674396] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6246.674879] LustreError: 292235:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6246.742620] LustreError: 294599:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6246.744342] LustreError: 294599:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 6246.803624] Lustre: server umount lustre-MDT0001 complete [ 6250.492836] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 6251.596807] Key type lgssc unregistered [ 6251.734449] LNet: 295130:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6251.736793] LNetError: 295130:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6251.745490] LNet: Removed LNI 192.168.204.143@tcp [ 6252.074510] Key type .llcrypt unregistered [ 6252.075624] Key type ._llcrypt unregistered [ 6259.897182] Key type ._llcrypt registered [ 6259.898214] Key type .llcrypt registered [ 6259.940952] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 6266.322267] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 6266.823823] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6266.867864] alg: No test for adler32 (adler32-zlib) [ 6267.729486] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 6267.828206] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 6269.415187] Key type lgssc registered [ 6269.807221] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6272.824193] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6274.782784] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6277.736746] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 6280.642410] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6282.933818] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6282.958055] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6282.962128] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6284.043210] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6284.055019] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6284.090128] Lustre: lustre-MDT0000: new disk, initializing [ 6284.116472] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6284.124077] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6285.517718] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6289.926695] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6289.949414] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6290.086110] Lustre: lustre-OST0000: new disk, initializing [ 6290.088539] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6290.091193] Lustre: Skipped 1 previous similar message [ 6290.116961] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6292.164401] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 6292.167297] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 6292.178129] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 6292.304663] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6471.649050] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6471.653461] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6472.012598] LustreError: 300044:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 6472.127452] Lustre: server umount lustre-MDT0000 complete [ 6473.365554] LustreError: 299093:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763159764 with bad export cookie 14190478411163652491 [ 6473.367343] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6473.370635] LustreError: 299093:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6479.467831] LustreError: 300246:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6479.469601] LustreError: 300246:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6479.518868] Lustre: server umount lustre-OST0000 complete [ 6487.997078] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 6491.311538] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 6495.232317] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6497.475929] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6499.576977] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6501.881815] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6504.231043] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6504.249061] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6504.339548] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6504.351083] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6504.390639] Lustre: lustre-MDT0000: new disk, initializing [ 6504.417760] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6504.424815] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6505.804175] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6509.986461] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6510.007861] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6510.039313] Lustre: 303283:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6510.063449] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6510.065417] Lustre: Skipped 1 previous similar message [ 6510.102213] Lustre: lustre-MDT0001: new disk, initializing [ 6510.137213] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6510.150593] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6510.154415] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6511.491505] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6514.636955] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6515.843600] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6518.110943] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6518.135913] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6518.253026] Lustre: lustre-OST0000: new disk, initializing [ 6518.254772] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6518.287674] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6519.866305] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6519.869554] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6519.904057] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6520.235787] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6523.350309] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6524.840521] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 6524.924112] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 6526.268974] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 6526.355842] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 6529.069091] LustreError: 306218:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 6529.071174] LustreError: 306218:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6529.107155] Lustre: server umount lustre-OST0000 complete [ 6530.528027] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6530.530294] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6530.534459] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6535.136612] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6535.137157] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6540.256855] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6540.260102] Lustre: Skipped 2 previous similar messages [ 6540.261981] LustreError: 306419:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 6540.264791] LustreError: 306419:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6540.293257] Lustre: server umount lustre-MDT0000 complete [ 6541.586228] LustreError: 303317:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763159833 with bad export cookie 14190478411166593268 [ 6541.589408] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6541.592087] LustreError: 303317:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6552.035395] LustreError: 306621:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6552.037221] LustreError: 306621:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6552.087989] Lustre: server umount lustre-MDT0001 complete [ 6553.238426] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 6553.772227] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 17:37:24 (1763159844) [ 6555.995949] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6556.179353] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6557.549350] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6560.461534] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6561.927637] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6563.698239] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6564.952081] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6567.236896] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6569.259345] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6571.075398] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6573.423729] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6573.444601] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6573.501457] Lustre: lustre-OST0001: new disk, initializing [ 6573.503341] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6573.539538] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6573.541422] Lustre: Skipped 2 previous similar messages [ 6575.523944] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6578.701887] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6578.707489] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6578.726452] LustreError: 310257:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 6578.731155] LustreError: 310257:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 6579.748837] LustreError: 310262:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 6579.752533] LustreError: 310262:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 6580.783293] LustreError: 310262:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 6580.786352] LustreError: 310262:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 6582.821374] LustreError: 310262:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 6582.824641] LustreError: 310262:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 6584.365902] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 6584.454186] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 6585.669601] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9cc1c3587000.ost_server_uuid 50 [ 6586.189542] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9cc1c3587000.ost_server_uuid in FULL state after 0 sec [ 6586.914589] LustreError: 310262:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 6586.920349] LustreError: 310262:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 6587.312587] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9cc1c3587000.ost_server_uuid 50 [ 6587.810356] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9cc1c3587000.ost_server_uuid in FULL state after 0 sec [ 6588.969132] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6601.189645] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6601.191081] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 6601.193345] Lustre: Skipped 1 previous similar message [ 6605.548087] LustreError: 311010:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 6605.549906] LustreError: 311010:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6605.568765] Lustre: server umount lustre-OST0001 complete [ 6606.307181] LustreError: 308792:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6606.311666] LustreError: 308792:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6611.428251] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6611.429463] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6611.429546] LustreError: 310264:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6611.429555] LustreError: 310264:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6611.430417] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6611.430421] Lustre: Skipped 2 previous similar messages [ 6611.443438] Lustre: Skipped 1 previous similar message [ 6613.325535] Lustre: server umount lustre-OST0000 complete [ 6620.694099] Lustre: server umount lustre-MDT0000 complete [ 6621.663821] LustreError: 310903:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6621.669548] LustreError: 310903:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6621.887086] LustreError: 308812:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763159913 with bad export cookie 14190478411166594136 [ 6621.891632] LustreError: 308812:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6621.893136] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6621.962068] LustreError: 311615:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 6621.965455] LustreError: 311615:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 6622.024850] Lustre: server umount lustre-MDT0001 complete [ 6625.335650] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 6626.328879] Key type lgssc unregistered [ 6626.465448] LNet: 312145:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6626.468097] LNetError: 312145:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6626.474322] LNet: Removed LNI 192.168.204.143@tcp [ 6626.778314] Key type .llcrypt unregistered [ 6626.779339] Key type ._llcrypt unregistered [ 6635.532833] Key type ._llcrypt registered [ 6635.533869] Key type .llcrypt registered [ 6635.568236] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 6635.973807] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6636.056784] alg: No test for adler32 (adler32-zlib) [ 6636.911337] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 6636.994454] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 6638.575104] Key type lgssc registered [ 6638.880806] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6641.214782] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6641.219979] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6642.375971] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6643.570416] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6645.275530] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6647.432770] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6647.551833] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6648.762850] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6651.835695] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6651.953974] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6653.870105] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6655.626846] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6657.063169] LustreError: 315075:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6657.073281] LustreError: 315075:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6657.874682] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6657.942836] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6659.818442] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6661.484673] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 6667.251739] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 6667.254545] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 6673.602409] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 6673.683101] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 6749.157291] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6749.158767] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 6749.158842] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6749.162571] Lustre: Skipped 1 previous similar message [ 6749.172384] Lustre: Skipped 1 previous similar message [ 6754.274451] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 6754.276172] Lustre: Skipped 1 previous similar message [ 6754.990080] LustreError: 316964:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 6755.004870] Lustre: server umount lustre-OST0001 complete [ 6759.394751] LustreError: 315077:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6759.394751] LustreError: 316521:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6759.395209] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6759.395962] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6759.396880] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6759.409285] Lustre: Skipped 1 previous similar message [ 6762.603123] LustreError: 317166:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 6762.606169] LustreError: 317166:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6762.645484] Lustre: server umount lustre-OST0000 complete [ 6763.937258] LustreError: 317367:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6763.940480] LustreError: 317367:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6763.981946] Lustre: server umount lustre-MDT0000 complete [ 6764.512576] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6764.513740] LustreError: 313518:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6764.516113] Lustre: Skipped 1 previous similar message [ 6764.524050] LustreError: 313518:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6765.166155] LustreError: 316865:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763160056 with bad export cookie 4814395420198344550 [ 6765.170666] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6765.171336] LustreError: 316865:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6765.288585] Lustre: server umount lustre-MDT0001 complete [ 6768.586954] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 6769.570854] Key type lgssc unregistered [ 6769.698715] LNet: 318099:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6769.702161] LNetError: 318099:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6769.713441] LNet: Removed LNI 192.168.204.143@tcp [ 6770.072687] Key type .llcrypt unregistered [ 6770.074297] Key type ._llcrypt unregistered [ 6787.325703] Key type ._llcrypt registered [ 6787.326660] Key type .llcrypt registered [ 6787.359699] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 6787.675867] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6787.732611] alg: No test for adler32 (adler32-zlib) [ 6788.587745] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 6788.670513] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 6790.247167] Key type lgssc registered [ 6790.603731] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6793.583319] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6793.591279] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6794.759680] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6796.134386] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6799.040312] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6799.183347] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6800.451603] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6801.427725] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6803.695281] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6803.844589] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6804.903879] LustreError: 321805:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6804.907560] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 6804.909143] LustreError: 321805:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6805.800214] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6808.615881] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6808.718289] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6809.781145] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 6810.745670] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6815.963479] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6820.545818] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6830.047992] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6830.051883] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6830.058733] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6830.563124] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6830.564247] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6830.569064] Lustre: Skipped 2 previous similar messages [ 6830.570541] Lustre: Skipped 1 previous similar message [ 6835.264683] LustreError: 323752:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6835.291885] Lustre: server umount lustre-MDT0000 complete [ 6835.680588] LustreError: 320344:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6835.685175] LustreError: 320344:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6836.487977] LustreError: 320325:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763160127 with bad export cookie 14392807264298074725 [ 6836.490337] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6836.490952] LustreError: 320325:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6836.557842] LustreError: 323953:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 6836.559491] LustreError: 323953:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6836.603488] Lustre: server umount lustre-MDT0001 complete [ 6847.987096] LustreError: 324153:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 6847.988716] LustreError: 324153:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6848.003691] Lustre: server umount lustre-OST0000 complete [ 6859.378703] LustreError: 324354:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6859.380382] LustreError: 324354:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6859.428184] Lustre: server umount lustre-OST0001 complete [ 6864.779992] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 6868.456218] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6868.635179] LustreError: 325530:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6868.660210] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6869.910891] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6872.689672] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6874.093067] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6875.072655] Lustre: 326638:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6877.302640] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6877.424661] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6877.426536] Lustre: Skipped 1 previous similar message [ 6879.329026] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6879.461778] LustreError: 326993:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6879.467783] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 6882.218375] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6883.306836] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 6884.238654] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6887.382093] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6888.578142] Lustre: 328487:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6898.658578] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6898.660733] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6898.661735] Lustre: Skipped 3 previous similar messages [ 6898.661983] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6898.667789] Lustre: Skipped 4 previous similar messages [ 6900.670583] LustreError: 328593:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6900.673688] LustreError: 328593:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6900.707262] Lustre: server umount lustre-MDT0000 complete [ 6901.885777] LustreError: 328490:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763160193 with bad export cookie 14392807264298076412 [ 6901.888707] LustreError: 328490:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6901.889046] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6902.025449] Lustre: server umount lustre-MDT0001 complete [ 6912.946079] LustreError: 328994:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 6912.949183] LustreError: 328994:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 6912.968336] Lustre: server umount lustre-OST0000 complete [ 6924.393562] Lustre: server umount lustre-OST0001 complete [ 6925.633750] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 6926.114814] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 6926.675807] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 17:43:37 (1763160217) [ 6933.789271] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 6937.028206] loop0: detected capacity change from 0 to 1099511627776 [ 6937.198439] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 6937.200885] *** VALIDATE xfs *** [ 6937.209359] XFS (loop0): Mounting V5 Filesystem [ 6937.221333] XFS (loop0): Ending clean mount [ 6937.735870] loop1: detected capacity change from 0 to 18691697672192 [ 6952.487539] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6959.250214] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 6961.957342] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 6965.358830] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6967.430734] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6969.351053] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6971.311400] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6973.549779] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6973.572947] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6973.651996] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6973.665689] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6973.710800] Lustre: lustre-MDT0000: new disk, initializing [ 6973.736772] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6973.738559] Lustre: Skipped 1 previous similar message [ 6973.743900] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6974.950690] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6979.013551] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6979.046342] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6979.087333] Lustre: 333563:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6979.117612] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6979.119606] Lustre: Skipped 1 previous similar message [ 6979.165859] Lustre: lustre-MDT0001: new disk, initializing [ 6979.220757] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6979.224540] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6980.579453] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6983.578894] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6984.684463] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6986.805341] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6986.822488] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6986.919705] Lustre: lustre-OST0000: new disk, initializing [ 6986.921494] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6988.536028] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6988.540901] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6988.571767] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6988.852912] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 6991.839206] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6993.100848] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 6993.168829] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 6994.434814] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 6994.507206] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 6999.009918] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6999.010802] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6999.012314] LustreError: Skipped 1 previous similar message [ 6999.018860] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6999.022109] Lustre: Skipped 1 previous similar message [ 7001.197927] LustreError: 336505:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 7001.199656] LustreError: 336505:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 7001.242306] Lustre: server umount lustre-OST0000 complete [ 7004.128701] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7004.129764] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7004.131810] Lustre: Skipped 2 previous similar messages [ 7004.135875] Lustre: Skipped 1 previous similar message [ 7008.532479] Lustre: server umount lustre-MDT0000 complete [ 7009.247973] LustreError: 334349:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7009.251834] LustreError: 334349:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7009.731179] LustreError: 334343:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763160301 with bad export cookie 14392807264298077966 [ 7009.734477] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7009.736440] LustreError: 334343:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7009.864098] Lustre: server umount lustre-MDT0001 complete [ 7012.499555] XFS (loop0): Unmounting Filesystem [ 7013.994245] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 17:45:04 (1763160304) [ 7016.094598] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7016.256870] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7016.259452] Lustre: Skipped 2 previous similar messages [ 7017.425865] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 7020.006224] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7021.253913] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 7022.858332] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7024.024714] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7026.127227] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7027.928352] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 7029.550447] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7051.234637] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7051.234665] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7051.242508] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7051.244075] Lustre: Skipped 1 previous similar message [ 7054.316987] LustreError: 340089:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7054.318703] LustreError: 340089:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 7054.356806] Lustre: server umount lustre-OST0000 complete [ 7055.839766] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7055.841648] LustreError: Skipped 1 previous similar message [ 7061.582821] Lustre: server umount lustre-MDT0000 complete [ 7062.745335] LustreError: 337705:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763160354 with bad export cookie 14392807264298078848 [ 7062.748841] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7062.751073] LustreError: 337705:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7062.870365] Lustre: server umount lustre-MDT0001 complete [ 7066.034767] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 7066.965815] Key type lgssc unregistered [ 7067.099311] LNet: 341023:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7067.101415] LNetError: 341023:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7067.110281] LNet: Removed LNI 192.168.204.143@tcp [ 7067.406094] Key type .llcrypt unregistered [ 7067.407514] Key type ._llcrypt unregistered [ 7076.664694] Key type ._llcrypt registered [ 7076.665634] Key type .llcrypt registered [ 7076.695814] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 7077.081606] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7077.088564] alg: No test for adler32 (adler32-zlib) [ 7077.927194] Lustre: Lustre: Build Version: 2.16.61_2_g297829f [ 7077.999921] LNet: Added LNI 192.168.204.143@tcp [8/256/0/180] [ 7079.575148] Key type lgssc registered [ 7079.912869] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7082.451381] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7082.458712] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7083.619365] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7084.808491] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 7087.525549] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7087.661975] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7088.944660] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 7090.682345] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7091.800697] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7093.993741] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7094.119971] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7095.993123] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 7097.636243] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7118.304053] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 7118.306712] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7118.310639] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7119.843888] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7119.845526] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7119.845873] Lustre: Skipped 1 previous similar message [ 7122.045783] LustreError: 344864:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7122.070992] Lustre: server umount lustre-MDT0001 complete [ 7123.010774] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7123.662045] LustreError: 344703:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.43@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7123.667362] LustreError: 344703:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7124.962592] LustreError: 342547:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7124.970108] LustreError: 342547:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7125.482067] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7125.515430] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 7125.517284] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7125.531092] Lustre: 342536:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7125.657133] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7126.929426] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 7131.110682] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 7132.644932] LustreError: 345991:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 7132.647102] LustreError: 345991:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 7132.650310] LustreError: 342540:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 7132.665374] LustreError: 345994:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.204.143@tcp: cfg command failed: rc = -17 [ 7132.667940] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 7132.667940] [ 7430.867021] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.43@tcp (stopping) [ 7433.185330] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7433.186093] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 7433.188968] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7433.192101] Lustre: Skipped 1 previous similar message [ 7435.260237] LustreError: 357781:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7435.261969] LustreError: 357781:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7435.284926] Lustre: server umount lustre-MDT0001 complete [ 7435.981727] LustreError: 342572:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.43@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7436.113922] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7438.305171] LustreError: 343918:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7438.309166] LustreError: 343918:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7438.399322] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7438.431616] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 7438.433252] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7438.446562] Lustre: 342536:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7438.559803] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7439.706279] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 7443.941295] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 7444.452671] LustreError: 358864:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 7444.454797] LustreError: 358864:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 7444.457112] LustreError: 342540:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 7444.459069] LustreError: 342540:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 7444.476968] LustreError: 358867:0:(obd_config.c:689:class_setup()) Device 8 already setup (type osp) [ 7444.478949] LustreError: 358867:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.204.143@tcp: cfg command failed: rc = -17 [ 7444.481340] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.143@tcp [ 7444.481340] [ 7743.191095] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.43@tcp (stopping) [ 7746.019728] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 7746.023083] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7746.034075] Lustre: Skipped 1 previous similar message [ 7748.426820] LustreError: 369896:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7748.429825] LustreError: 369896:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7748.461818] Lustre: server umount lustre-MDT0001 complete [ 7749.685786] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7751.139620] LustreError: 343918:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7751.144897] LustreError: 343918:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7753.141137] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7753.192271] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 7753.195501] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7753.216140] Lustre: 342536:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7753.371679] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 7754.949593] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 7758.831495] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 7759.335168] LustreError: 370967:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 7759.338754] LustreError: 370967:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 7759.342662] LustreError: 342540:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 7759.345519] LustreError: 342540:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 7759.370758] LustreError: 370976:0:(lod_lov.c:207:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 7759.382328] LustreError: 370976:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.204.143@tcp: cfg command failed: rc = -114 [ 7759.386358] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 7759.386358] [ 8071.145057] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8071.147754] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8071.152037] Lustre: Skipped 3 previous similar messages [ 8071.155317] Lustre: Skipped 4 previous similar messages [ 8071.160661] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8076.047833] LustreError: 381994:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 8076.049948] LustreError: 381994:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8076.086938] Lustre: server umount lustre-MDT0000 complete [ 8076.258547] LustreError: 342542:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8076.268169] LustreError: 342542:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8077.879724] LustreError: 345428:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763161369 with bad export cookie 2835558554478321339 [ 8077.888237] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8077.888549] LustreError: 345428:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8081.378256] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8084.053099] LustreError: 382196:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 8084.056856] LustreError: 382196:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8084.133928] Lustre: server umount lustre-MDT0001 complete [ 8086.213212] Lustre: server umount lustre-OST0000 complete [ 8090.801187] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 8095.609827] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 8100.599696] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8103.883907] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8106.779463] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8109.905901] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8112.771743] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 18:03:23 (1763161403) [ 8115.302051] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8115.324298] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8115.425261] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8115.436469] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8115.472364] Lustre: lustre-MDT0000: new disk, initializing [ 8115.500977] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8115.506672] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8116.814670] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 8120.776095] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8120.800673] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8120.828724] Lustre: 384927:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8120.852365] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8120.853870] Lustre: Skipped 1 previous similar message [ 8120.886759] Lustre: lustre-MDT0001: new disk, initializing [ 8120.919506] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8120.930918] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8120.933502] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8122.078244] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 8125.176122] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8126.387200] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8128.372019] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8128.392108] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8128.478258] Lustre: lustre-OST0000: new disk, initializing [ 8128.480048] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8128.504177] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8130.224121] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_default_debug -1 all [ 8130.677357] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8130.679936] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8130.709590] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8132.966439] Lustre: DEBUG MARKER: oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8136.161891] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8136.163243] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8136.164922] Lustre: Skipped 2 previous similar messages [ 8136.166412] Lustre: Skipped 4 previous similar messages [ 8136.168598] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8141.499337] LustreError: 387372:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 8141.500909] LustreError: 387372:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 8141.524790] Lustre: server umount lustre-MDT0000 complete [ 8142.622897] LustreError: 386674:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763161434 with bad export cookie 2835558554478323292 [ 8142.625878] LustreError: 386674:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8142.626612] LustreError: MGC192.168.204.143@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8142.626647] LustreError: 385719:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 8142.726601] Lustre: server umount lustre-MDT0001 complete [ 8154.075231] Lustre: server umount lustre-OST0000 complete [ 8164.949513] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing set_hostid [ 8169.065494] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing load_modules_local [ 8173.474575] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8176.163677] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8179.070297] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8181.993935] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8182.903282] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 7943 sec ========== 18:04:33 (1763161473) [ 8183.658123] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 18:04:33 (1763161473) === [ 8184.846991] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 18:04:35 (1763161475) === [ 8200.204230] Lustre: DEBUG MARKER: oleg443-server.virtnet: executing unload_modules_local [ 8201.513609] Key type lgssc unregistered [ 8201.665622] LNet: 392879:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8201.670291] LNetError: 392879:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8201.685548] LNet: Removed LNI 192.168.204.143@tcp [ 8202.135873] Key type .llcrypt unregistered [ 8202.137974] Key type ._llcrypt unregistered